nthExperiment

by Prabakaran Chandran

An Introspection

I've spent my career thus far working on a range of challenges in data science and machine learning, driven by a genuine curiosity to tackle interesting problems faced by enterprises and startups.

I believe that, over time, I've managed to do some meaningful work. I vividly remember my early days at Mu Sigma, where a forecasting project initially involved months dedicated solely to data preparation and automating the validation and reporting of model performance. Later, I gained opportunities to work on truly end-to-end problems.

My trajectory over the years feels quite different from that of many peers. For most, the transition into the field of DS and ML seems driven by hype or influence. For me, starting with a background in instrumentation and control engineering, it felt like a natural progression. The Mu-Sigma ecosystem was instrumental in helping me develop a "problem solver DNA" rather than a "skill stacker DNA." The former is fundamentally about adopting a problem-first mindset and diligently putting in the effort required to solve it. The latter, in my view, is more linear and often keeps learning somewhat detached from real-world application; it tends to prioritize accumulating courses, tutorials, and certifications. I don't mean this in a negative way – the skill-stacker mindset is perfectly suited for individuals who wish to settle into a specific job role and excel within its defined boundaries.

The Restless Journey

However, I've always felt a restless urge to go beyond simply doing what seniors, social media influencers, or companies might demand (like chasing specific certifications or spending days on online courses), even though I engaged in some of these activities early on. My consistent aim has been to steer my journey in a way that allows me to develop my own thought process, explore ideas, and actively try things out.

Looking back over the past six years, it feels like a true parallel journey. On one track, I worked within organizations, addressing their specific problem statements. On the side, I embarked on a "side hustle" of launching and failing at numerous experiments. These ranged from designing a course focused on fundamental learning (inspired by approaches like Stanford's, emphasizing concepts, experimentation, and teaching) under the name "beyond-dotfit," to trying to build a platform for collaborative real-world problem-solving, and even a series of micro-SaaS ideas. They all failed, and perhaps they deserved to, as they didn't fully align with the intended audience's needs. Yet, every single one of these efforts was deeply introspective and moved me forward, always leading to a new layer of learning.

I could easily have started a YouTube channel focusing on standard "how-to" or "what-is" content, like many peers. But I always questioned: why should I do this? What unique value could I bring? It's true that without trying, one can't definitively answer that, and perhaps I prematurely abandoned the idea – something I feel ambivalent about.

Over the years, I've found it challenging to confine myself to a single horizon or vertical domain. I recall one manager introducing me to the leadership team as a "horizontalist," explaining how I assisted teams across the Business Unit with problem-solving. I genuinely enjoyed moving between teams, getting involved in diverse tasks ranging from designing proposal decks and deploying models to mathematically validating assumptions, and much more. My role at my second company was also highly horizontal; my manager and I tackled everything from entire product development and data science practice to running deep learning experiments – a true 0-to-1 transition experience.

Given this background, I feel a strong pull to continue nurturing this horizontalist, generalist problem-solving mindset. I believe this approach is increasingly vital for individuals, organizations, and institutions. It addresses the gap between those who might learn to build advanced AI agents but lack the understanding of their true business application, or Product Managers who excel in their field but struggle to conceptualize a problem in a mathematical or AI-driven way. While opportunities to apply this mindset fully might not always present themselves in traditional employment, I've increasingly found such chances outside of work, reinforcing that valuable problem-solving isn't confined to a job description. My innate curiosity, interdisciplinary interests, resourcefulness, and overall mindset continually drive me to pursue this path.

What's Next: nthExperiment

So, what's next? Perhaps this is the first step in creating the ecosystem that I've been seeking. If I can't consistently find the ideal environment offered elsewhere, I have to build it myself. This isn't the first time I've attempted something of this nature, but this time I have a clearer plan and envision a space where learning, 'geekdom,' and ultimate resourcefulness intersect.

Thus, nthExperiment will be the umbrella under which I conduct, present, and collaborate on experiments in Data Science and Artificial Intelligence (AI) Engineering. I currently perceive this domain as "Decision Engineering" (though I'll use this term sparingly until the scope and outcomes become clearer).

In essence, nthExperiment is an experiment in itself – a way to validate ideas like self-designing one's own learning journey, building personal "labs," and prioritizing problem-solving. I use the term "labs" not in the formal research sense, as rigorous research requires specific definitions and structures, but in the context of individual effort and freedom to explore and solve problems.

Moving forward, all my efforts and learnings will be consolidated under this single umbrella, nthExperiment.

I also plan to hold short presentations every Thursday, sharing the outcomes of experiments and discussing foundational knowledge.

I have the idea, the energy, and a strong purpose to make this happen! It will be an iterative, self-organizing process.

As a first step, I'm focusing on core machine learning and statistics concepts. My initial experiment will compare the generalization capabilities of SVMs and Neural Networks, partly driven by my interest in Professor Vapnik's work. A second experiment will explore "Reinforcement Learning with Statistical Models." My curiosity here stems from the common assumption that RL must start with Q-learning; I wonder if Q-values could be learned using statistical models instead?

I plan to document these experiments as I go – a practice I've learned over time. This involves capturing the questions that arise naturally from my 'System 1' thinking. Please let me know if any of my questions seem naive!

Keep experimenting!

Future Ideas

This page is reserved for future thoughts, experiments, or updates.

Stay tuned!

Timestamp