A common problem many of us face is deciding what to work on next. It happens to everyone: too many goals lead to analysis paralysis, and too few goals lead to apathy. Drawing from my past experience working on recommendation systems, there’s a concept called "Explore and Exploit" that I believe is applicable to life in general.

What is Explore and Exploit?

Imagine Netflix trying to figure out what shows you like. That’s a “cold start” problem. When you first log in, Netflix doesn't know your preferences yet, so it shows you a variety of popular genres (around you) —action, documentaries, rom-coms—to gauge your interests. This is the exploration phase.

Each exploratory content has a possibility of converting into exploit content. Netflix selects the most probable content first.

As you start watching and binging, Netflix begins to understand your preferences and focuses on showing you more of what you like, maximizing your engagement. This is the exploitation phase.

The main goal of exploratory content is to create a pathway for new exploit content.

The Psychology Behind It

This strategy has been exceptionally successful for the tech industry, and virtually every recommendation system in the world uses it. Daniel Kahneman, a Nobel laureate, talks about our need to balance novelty with mastery. We enjoy discovering new things (exploration) but also like sticking with what we know and refining it (exploitation).

Striking a good balance can make our lives more effective and satisfying. In fact, one of the things a good recommendation system does is balance exploratory content with exploit content so you never get bored with the system.

Exploring yourself with side projects

I used to think that side projects were a distraction. Why wasn’t I focusing on the main thing? If you have multiple commitments, you might have had those feelings of guilt too. Should you really be building that robot or building that tool when you should be focusing on your job?

Continuing the analogy of recommendation systems, if the system focuses solely on exploitation, users eventually get bored and leave. Humans need novelty. In the real world, we get bored too; if we can't bounce off due to commitments, we get burnt out.

Based on my personal experience, here's my advice for people who like trying out multiple things:

Give Yourself Permission to Explore

Label Your Projects: When picking up new projects, label them as either exploratory or exploitative.

Exploratory projects are easy to abandon. If you pick up something and hate it, just abandon it.

Abandoning exploit projects is a high-cost decision. You're probably leaving behind something valuable—money, stability, or something important. Know what you're giving up before abandoning it.

You Might Belong to Two Categories

Too many goals

Chances are you’re just curious to try them out. It’s extremely unlikely for all of them to stick. Instead of feeling guilty about pursuing them, I highly recommend time-boxing these exploratory projects.

Start with 20-50 hours, depending on your expectations. If you enjoy it during that time and it keeps you engaged, continue with it. Otherwise, discard it. Over time, you’ll get a sense of what you enjoy and what you don’t, allowing you to pick projects with a higher possibility of converting to exploit mode.

Too few goals

Some of us don’t know what to pick because we’re unsure of what we might enjoy. The process is similar: you're probably in cold start mode. Look around at what others enjoy. Pick up some of these interests, and follow the same loop as above.

The Balancing Act

Exploration and exploitation are not one-off activities; they are ongoing parts of life and need constant balancing. Some periods may necessitate exploitation, while others give you the freedom to explore new interests. Recognize the period you’re in and balance your time accordingly.

I hope this analogy made sense and that you find it useful in navigating your projects.