Building at Clear Street

At Clear Street, we are upgrading the infrastructure that powers global trading. It’s a part of the financial system that hasn’t seen innovation in decades.

Later this week, we will launch an update to one of our own infrastructure services. We interviewed Tanishq Dubey, who led development of the program, to learn more about the problem, the product, and how he built it.

Q: You’re launching an update to Reference Master next week. What is it? Why is it important?

Reference Master is a program that we use internally to help manage global trading. It basically holds all the information you’d need to trade any instrument in any market around the world. It contains all the instruments sold in every country, the current price, and a bunch of other information that traders need.

It sounds easy — it’s basically just a big dictionary — but the challenge is that every country has its own way of naming stocks. For example, in the US, Apple trades as AAPL. In London, it trades as OR2V. In Germany, it trades as APC or 4033819. Multiply that times all the instruments on Earth and the problem gets big pretty fast.

Q: This sounds like it would be a fairly common problem in the industry, right?

Yes, any company that buys and sells stocks globally experiences this problem.

One of my teammates worked for a large bank before joining Clear Street. They solved this problem manually. They hired about 50 people overseas, who would update the information on every stock, in every market, every day.

There are a number of similar infrastructure problems that banks and system providers have solved with brute force. It’s effective in the short term, but it’s expensive, inefficient, and limiting in the long run. And, as an engineer, it’s incredibly frustrating to work with.

We’re taking a different approach at Clear Street. We’re solving big infrastructure problems elegantly. We’re building for speed, scale, and innovation. We’re building for the future.

Q: How did we solve this problem?

It’s been an iterative process. We’re actually launching the next generation of the system this week. The first version parsed a massive, multi-gig file every night. Whenever we found a missing symbol, it would take a day to fix the problem, which disrupted our operations team.

The new version is a service. When our operations team enters a symbol that doesn’t exist in our system, the program finds the missing symbol, validates it, and updates the system instantly. Operations never even knows the symbol is missing.

Q: That’s quite a generational leap. How do you feel about the progress?

Honestly, it’s made me feel really confident about the software we’re writing.

Let me explain why.

We’re testing the new program now. Whenever the system crashes, it generates an error and alerts the team. Every time there’s been an error, we’ve traced the problem back to a third party with the best data set available on the market, one that comes directly from a major exchange.

In just a few months, we’ve gone from parsing a multi-gig file to building a service that’s identifying errors in our source of truth. That’s progress.

Q: Sounds like it. Let’s switch gears for a second. Did you have experience with the markets, when you started at Clear Street?

No, I didn’t. I had Robinhood on my phone, but that was the extent of my exposure to financial markets. I had no idea these systems existed behind the scenes.

Luckily, we have a great mix of people at Clear Street, who come from different backgrounds. Our operations team has decades of experience building and operating large-scale clearing systems.

It’s helpful to have them build with us. It’s been a huge learning opportunity and a very rewarding experience, overall.