A guide to R&D

A little over a month ago, I announced that I had switched positions at my current employer.

With our new funding round announcement, I am also taking a new position at Coalition as Head of R&D. I'm grateful that I got to run+grow the @binaryedgeio teams these years, I now shift my focus to build an even larger moat for @SolveCyberRisk when it comes to risk selection.

— Tiago Henriques (@Balgan) July 8, 2022

I had already been running the research team at Coalition for a while, however on top of that I was running Security engineering operations (my old startup BinaryEdge, Coalition Control, Security analysts, etc…), which in itself is easily a fulltime job. So with us having found someone fantastic to run the SecEng ops, I’m now moving to focus full-time on R&D.

This post is a write up on how I think about R&D, the team and approaches that my teams will take.

The job…

Often when thinking about R&D teams, people immediately think about them building amazing technology. However, the research starts months if not years before we get to that point!

It all starts with customers!

Having been certified in the school of design thinking from IDEO, I’ve long been taught customer is king and should be your main focus.

So what is the job of a researcher? If I had to break it down, I believe it looks something like this:

(this process doesn’t have one direction and is very agile-ish in that you can iterate through multiple phases ie: reading <-> thinking <-> reading <-> documenting)

What do these different phases mean?

Observing - A researcher, like any other person that builds or wants to solve a problem, has an end customer. It can be internal or external, technical or non-technical among many other types (defining personas is part of a nice team exercise, a blogpost for another day). The way to understanding the problems you’re solving and finding new problems to solve, is to observe these users. Ask them to let you observe them while they work, or while they go on their day to day and document everything that you observe. The richer the documentation the better. For example, I’ve done exercises where we would record video of full days of work or following our customers as post-analysis of those videos allowed me extra insight that I couldn’t get immediately at observation time. However notes or voice recordings of your comments also work (hint: otther.ai provides a free service that transcribes your notes for free)!

Reading - A big part of being a researcher is knowing you don’t know everything and being open to learn new things. It’s important to keep up with what is happening in the world of research. Make sure you’re taking the time to look at Arxiv or journals to see the latest papers published. Diagonally read through many, filter down some to read in detail (specially those in areas of interest of you or your colleagues).

A great combo here is:

• Step 1 - Go to Open Knowledge Maps and type the topic you’re researching. This will give you a view like the following image, which looks for adjacent topics to yours and associated papers

• Step 2 - Select some papers you like, grab their Arxiv link for example https://arxiv.org/abs/2102.05568

• Step 3 - Go to https://inciteful.xyz/ - Type the link there and find similar papers

Thinking - You’re being paid to solve problems. But if the problems were easy to solve, someone else would have done so already. Take this time and really think about the observations you’ve made, formulate potential experiments and also try different thought models. Think about the second and third order impacts of some of the decisiosn you’re making when devising potential solutions.

My recommendation here is Miro. Miro is a fantastic tool, I am a heavy user and do a ton of visualizations, diagrams and workflows in Miro for some of the work that I am doing. (You can even see a diagram from this blogpost there!)

Executing - This is where you BUILD. Building can come in different forms. It can be writing code for an analysis, building a machine learning model, building a PoC or a Prototype (they are not the same.)

The execution on R&D teams tends to be very different from engineering teams, where you want to develop good maintainable, readable code and with tests. R&D is about failing fast, failing cheap, and recovering even faster. So we don’t typically get enamored with our code or even with the first 1-2-3 iterations of the solutions we find. It’s about speed and cutting down on the different experiments you’re trying out to filter down to an acceptable amount of final solutions.

Documenting - Writing things down should come natural to you; if you’re doing it along the other phases this phase should actually be more about “cleaning up” and organizing your documentation rather than actually doing full write ups. Your team should have some templates for projects that you can use and those should set expectations on the documentation needed per project.

My current recommendation here is Notion. I’ve tried a few others, but the flexibility of Notion, allowing sharing and publishing and being markdown based, make it my favorite tool.

This phase also becomes important as you think about handover. Typically an R&D team will find a solution for a problem that works and that is scalable, and then deliver it to an engineering team for final implementation and operations. Having clear documentation on how something works, how and why it was built will be insanely important.

Teaching - You will make mistakes, you will have successes, you will find novel research and problems. It’s extremely important that you hand over all these learnings to the rest of your team. Make sure you’re spending time demo-ing the stuff you’re building, sharing that cool paper that you just read/found, letting other people in the team know about that cool python function you just built and that they can re-use. Also, go learn from others.

Team

A team should have a diverse set of skills and methods of thinking. I wouldn’t want to create an R&D team where everyone has the same skillset (eg: software engineers) and same way of thinking (eg: machine learning models > rules engines). Different problems require different solutions, and different solutions require different schools of thought.

You will then also want to look to the background of the people you’re hiring and bring people that went to different colleges, have different backgrounds, while at the same time there is a small set of skills that they must all have in common: creativity (obvious), perseverance (the problems you’re solving shouldn’t be easy or there would be no need for an R&D team) and passion about the problems they are working on.

When defining my team, I looked to the type of problems I at least knew we were going to see and solve on a daily basis on the job, and from that I pivoted to sets of keywords that are adjacent to the area of the initial problem (eg: Insurance risk -> automation, big data, actuarial knowledge, data science, machine learning). These word clouds then define the different skills sets and weights that I need from the people in my team.

Thinking about which projects to tackle

Companies that get to the point of wanting an R&D team usually have a substantial amount of problems that can be tackled and, much like engineering work, the number of researchers to work on them is usually “not enough”.

To decide which my teams tackle and prioritize them, any problem that is discovered is placed in a quadrant that looks a little something like this, which can also then be to give status updates to stakeholders:

In this quadrant I present the Risk and Reward of the different projects but also the “state” of each milestone, in a clear and easy way.

One other interesting piece about placing them in a quadrant like this is that there is typically a correlation of higher risk/reward with project complexity and, depending on availability of your team members, it becomes easier to allocate the work (for example, you might only have new joiners or less experienced members available that can only tackle specific projects, vs more senior and experienced that can attack more complex projects).

Conclusion

R&D teams tend do operate at a different speed from engineering teams, however the demand in documentation and clarity from them should also be extremely high, I believe R&D teams can learn a lot from Sr PMs by observing how they write requirements for engineering teams, to understand what type of documentation they should be producing on the projects they are working on.

An R&D team can’t always be working on moonshot projects, there are sometimes some smaller short term (in R&D terms thats 6 months to 1 year), that can and should be tackled, and this mostly has to do with time. Often due to lack of resources engineering teams aren’t offered the benefit of researching and being able to test different solutions to problems, so the R&D team being an extension to help engineering teams by tackling that is a positive.