Why Datasite Stopped Building Its Own Parsing - Adit Abraham, Reducto & Raj Bakhru, Datasite

[music] I'm personally really excited about this talk. Uh just by way of conro, I'm a de I'm co-founder and CEO at Redto. Uh but most importantly we have Raj here who is the global chief technology officer at data site. Um the reason why I'm excited uh is I think Raj has probably one of the most interesting viewpoints for AI and that he founded a company very early on in the vertical AI space. Um got acquired into data site and then got to see both like subsidiary within datasite to thinking about data as a whole. Um I got some of Raj's stats before this talk. Um where something like 60% of all transactions over hund00 million happen through data site. Is that right? >> That's correct. Yeah. >> Okay. Um, if we're making a leaderboard trillion plus tokens per year, I'm sure you have plenty of other cool numbers. Um, but the goal for today is to make this conversational. We'll talk a bit about Raj's journey, um, some of the problems he's gone through, and then we'll try to open it up for Q&A as well for whatever you all are curious about.

Sound good? >> Cool. Um, maybe to start, do you want to just walk through that arc and what it looked like behind the scenes? >> Yeah, for sure. Appreciate that. Um, good to meet everyone. Raj Bakru. Uh, my background is as a technologist. So, I spent my entire career in development. Started my career at Goldman Hybrbridge, KPOS Capital, was building research and trading systems for quant funds and all sorts of fun things. Started a security firm that I sold to a private equity back firm. Ended up running M&A and corporate strategy for them. And then started this vertical AI solution called Blue Flame in the M&A space. Um, because we sort of knew that the impact to unstructured data was going to be pretty pretty big. Um, so we were building out for about a year, year and a half and then data site approached us to acquire us. Um and we started working with Reductto actually while we were separate. Um so before data site had acquired us. Um we were a small startup. We were scrappy. Uh we built a lot of our own parsing and doc pipeline and ingestion pipeline. Um had a great team, a great CTO, co-founders. We were trying to save every dollar we could. Um so we when we built all of that, we were like this is great. It's working until it didn't. And I'll talk a little bit about that journey. Um and then as we got brought into data science and we were acquired obviously very early on. Our team was about 50 people at the time we got acquired. Um the first thing that data site wanted to do with us was bring us into all of the data rooms. And if you guys aren't familiar with the data room concept um data rooms are basically where all of the transaction documents

get stored when you're selling a company. So as a deep mentioned something like 60% of all transactions above a a reasonable size threshold 100 million happen on data sites. It's a place where people bankers primarily are sharing all of that data with private equity firms. Um so it means it's there's a massive unstructured data on data site right billions of pages of content. Every major bank is using it. Every private equity firm is using it. Every diligence provider is using it. Over 500,000 monthly active users on data site accessing that content. Um and that meant that adding AI solutions was obviously top of mind for the board and everyone else who was accessing the platform. users were asking for it all day in day out. We had a great data science team. Um, from the days where my CTO, co-founder and I sort of scrappily built the ingestion pipeline, we had obviously moved on to Reductto, recognizing that we weren't going to have the level of quality, speed, um, recall cost that we could get with Reductto. And I can talk about some of the numbers that led us to that that decision. Um and why that pans out today with data data site scale, but we needed to build fast. We needed to get all of the data rooms that data site had 55,000 data rooms a year um parsed ingested into our vector database so that we could build out our graph rag platform and answer questions within the data room uh that users were submitting at scale. Um so that was a problem at hand.

How do we how do we build this into the VDR? They had never done anything like this. They had never taken their data, their content out of their Azure platform. Historically, it's, you know, the walls were high around the data rooms. As you can imagine, it's the world's most sensitive data. All of the most material material non-public info out there sits in in data sites, virtual data rooms. Uh, so we had to figure out a way to do this securely. we had to figure out a way to do it um in a place that it uh complied with all the regulatory compliance regimes that our clients have and pass on to us, all of the certifications, the ISO certifications and so on that that we had. Um so we spent a lot of time with the deeds team and figuring out how to get there, how to make sure that all the content remained protected um and secured and get us all comfortable with leveraging uh Reduct's tooling. The reason that we went with Reduct's tooling as I sort of alluded to is we basically had to one move fast but two we did a whole bunch of benchmarking actually with our data science team around sort of our scrappily built tooling for extraction and parsing and ingestion and what I'll say about sort of that landscape is that um it's all edge cases like literally is everything is an edge case I remember getting you know after we had built this we were working with some of the world's largest private equity firms and they would send to us um examples where our rag platform got a question wrong. And actually, Carlos here will will remember some of these. [laughter] You'll dive into it and be like, well, I

got it wrong because the shade of blue on the legend of this pie chart didn't correlate perfectly and the LM just read it wrong, even though we used the best vision model in the world. And actually, if you were to pass that same thing as a pie chart or that pie chart um as a bar chart to that same model, it would have gotten it right. It just the devil's in the details on all of this stuff. um on Excel sheets we had to deal with merged cells and as you guys know finance spreadsheets are the most complicated documents in the world like it is hard for a human to understand these things um so getting the models to actually understand the way to parse this this content out and understand the content was very very difficult um so we found that we were just spending an enormous amount of support time dealing with all these edge cases and if I sort of think about the buy versus build trade-off what do I want my limited set of highquality engineers working on it is not the support cases of making sure that we're extracting extracting content properly um when that's already been solved like you know rebuilding the wheel doesn't make sense. Um, so we we got a tremendous amount of benefit and leverage out of saying let's let's pass that on to Reduct. Let's let our engineers work on value ad revenue generating business problems. Focus on building agents that our clients care about executing on the interface and the rag platform. Uh, building out graph rag was actually a big thing for us and we leverage the outputs of reductto in that process. So we wanted our data science team focused on that, not on how do I

get the right data out of a spreadsheet and dealing with all the pains that we experienced on that. Then you you run into the pains of like if it's an XSLB file um macro enabled, if it's a template file, it just it's endless. Literally the end edge cases are endless and we're very thankful to Reductive for a lot of those problems for us. >> I appreciate that. Um, at this point we've had like a year of context, but I think one of the most interesting things about your journey is that you were very early to the vertical AI app space. Um, and the problems that you work on are hard. Um, for many of the reasons that you mentioned, but also like things beyond that. Um, and especially earlier with the model landscape as it existed, I'm sure it felt even more insurmountable. Um, so I'd love to start maybe just by hearing a bit more of the pre-dataite journey of like how you got started with blue flame, what you were hoping to achieve. Um, and how joining data site has kind of changed that. >> Yeah. Um, so when when we launched our our startup startup, we had sort of we had sold a startup very early on before.

The last startup cyber security startup we sold a year in and we started this one saying we are not going to sell quickly. we are going to see this out [laughter] and it's going to be a long time and we're going to be a unicorn and do all those fun things that we wanted to do. Um, so that was that was the initial state. I brought in a lot of folks that we had worked in worked with before and that's one of the nice things of being a second time uh founder is you kind of know the network. You know the people you want to bring in. Uh so we were very fortunate to bring in a lot of those people and start with a lot of the people that we knew and trusted. Um and we could yell at each other and still be really good friends at the end of the day. Um and we knew each other's families and all that. So that really nice experience to start as a second time. Uh [snorts] we also had the ability thankfully to fund ourselves to a large extent this this second time. Uh so we bootstrap for a while and started building. Now that again made us do some things that might have been a little excessively cost-sensitive. Um and not the most optimal decisions but that's what you do as a startup um and wanted your own money especially. So we had a really great experience building that out. Uh we were actually going to do our first VC round. We had our LOI signed term sheet signed. Um, we had done sort of the dating process with all the VCs out there, the Sand Hill Roadwalk and all the New York ones. Um, and we were, you know, a week away from sort of final final signatures. And there was additional capacity on our round. Um, and they said, "You should go, you know, see if these folks are interested." So we we approached data science and we're like, "Do you guys want to put in 5 million in this round?" And we demoed and showed them a bit of what we were

doing and they said to us, "Actually, we just want to buy the whole company." Um we're like, "Okay." And literally happened over dinner. Um like whatever meeting in the office, subsequent dinner that night, and they told us that. Um so we we very quickly sort of figured it out. And the reason that we thought it was really compelling um was if you know, this was about a year and a half ago now. At that point, sort of claude and office suite hadn't taken off. You sort of have to go back to remember where we were a year and a half ago. It feels like forever ago, and the world was entirely different. Um, agentic coding hadn't taken off to the level it has now. Um, so we knew that we had to raise a massive amount of money. Our competitors in the vertical AI space, um, some of whom work with Reduct, had raised hundreds of millions of dollars, literally. Um, so we kind of knew that if we're going to remain competitive, we were going to be raising hundreds of millions of dollars. we would be taking masses mass dilution and doing that. Um, we also knew that staying competitive against the frontier models was going to be increasingly difficult, particularly as they were starting to launch vertical strategies. And at that point, sort of open AI for FS and Cloud for FS hadn't been fully announced, but there were some rumors around it. They'd made some some in insinuations about it. Um, and we knew that just to remain competitive and differentiated that we would have to bring in new types of data. And to us, the VDRs were this incredible valuable resource of prop data that had never been opened up to anyone in the world.

And we would be the only ones with access to it at that time. So that was super compelling. Um, data site also owns a private company database firm called Grata. Has about 20 million private companies in its database. Has transaction data, contacts, etc. used very heavily in private equity and banking worlds for um sourcing and business development origination to help me find add-ons from my platform portfolio company uh very classic grata type query that data would be brought into blue flame as well. So now we had these two really nice proprietary data sets feeding us in a vertical AI space where no one else would have that really nice and differentiated and that's played out really nicely. So over the over the course of the last year and a half, we've been bringing those really tightly together and um that's [snorts] that's been a really nice story for us to combine engineering teams, combine data, combine solutions. Um and we actually benchmark that like the the ability or the the fact that we brought the data closer to the models saves on token costs, it saves on run times um and increases quality of what we're doing. So that's been a that's been um quite a nice thesis to see pan out. um all of that data obviously requires scale. So that's that's again where reductive comes in and helps us with a lot of that ingestion processing and and so on. Data site over the course of the last year and a half since we've joined them has grown I want to say about 40% topline. Um so just the amount of content that's coming into the platform is is growing pretty quickly and pretty

massively. >> Um the demands on our AI solutions have gone up. So now people aren't just saying I want to ask questions against documents. the agents are obviously iterating against those documents doing much more fanciful diligence analyses or generating PowerPoint decks, IC memos and so on. Um, so it's a pretty wholesome suite of what they can what they can accomplish with the solution together. >> Yeah. Um, it's an interesting position to be in in that I think you have one of the most compelling data sets in the industry. Um, you're in a position now where obviously it's a global workforce. We were talking about all the different offices and all of that. um and model capabilities have gotten a lot better. Um so more broadly like how do you think about what you invest in internally versus what you build uh and buy externally as well. Um and also just like what sorts of things are top of mind as you think about the coming year? >> Yeah. Um great question. So we think a lot about the build versus buy, right? We we work with a lot of vendors. We spend a lot with a lot of vendors. Um and that's an intentional decision. Um, if [snorts] I think about what I want my my most talented people working on, it's not things that vendors can easily handle and are dedicated to handling and solving every day. Um, it's really hard to find really high quality good engineers, right? The fact of the matter is we're everyone in market is trying to hire the top tier talent of engineering talent right now because what we're seeing what you guys are all probably seeing is with AI coding the folks who get the most leverage and value out of

AI coding are the folks who have really strong mental models of code bases really understand engineering principles have done it for a long time um so they've all been 10xed you know over their normal 10x uh with AI with agent coding the middle layer s So um the bottom layer is actually getting a lot of leverage and now they don't they don't sort of uh have the burden of trying to figure out documentation that doesn't exist in all of that. The LMS can build that dynamically for them. Um so we're seeing sort of a barbell effect where we want to hire really really strong talent and really junior talent and build that up. Um where I want those people working is obviously not on support cases. It's not on things that are already solved. Um it's on the hardest and most pressing problems for the business at our scale. So data sites not a billion dollar topline but general range of a billion dollar topline. Um at our scale to grow 20% a year which is our ambition our board's ambition. Uh we need to be generating $200 million of revenue incremental revenue every year.

And if you ask yourself how are you going to go do that? Um, it's going to be by putting new AI products out in market, leveraging getting on top of token flow, expanding solution set that you're delivering to your clients. Um, reinventing the wheel is certainly going to get in the way of that. Um, having internal teams trying to uh iterate on some of the existing solutions that are already out there and deal with the support burden of those is not the best use of my talent. I have about 250 engineers in the org. So if I allocate out where I want those 250 focus, we have five major initiatives across the firm. Um those are five revenue generating projects that we're putting out, new new builds or consolidations of tools, new commercial or monetization models. Um I want those 250 fe people split across those five and I want as few working on the layers underneath that as possible. And if I can outsource everything on the under the layers underneath that, that's an ideal um ideal split of time for my my team.

>> Yeah. Uh before I turn over for questions, I realize we have two minutes. I guess maybe one last thing is like where do you find friction today? Um what are problems that you would hope people in the audience would solve for you? >> Um we still struggle with getting the models to do exactly what we want all the time. um we do a lot of benchmarking of all of the models and actually you know if you go back to pre-reduct world we wasted a lot of time on trying to figure out the right model for the right type of chart for the right type of uh document that we're passing in. So, we've we've gotten rid of a lot of that, but we still deal with it on the agent front, trying to benchmark out, you know, why is 61 soul tool spam, you know, tool call spamming when OBS 55 is not, but then 61 soul is so much cheaper that we want to move in that direction and we're trying to get it to go to go there. Um, so we do a lot of sort of the optimization of the three factors we care about, which is cost, speed, and quality. Quality is top top of mind for everyone in our space, right? like any finance user will pay any amount of money to have the best answer. Um so we need to make sure we're always optimizing on quality while trying to solve the other two.

>> Yeah, makes sense. Any questions from the room? >> Yeah. >> Can you zoom a little bit about the integration between data and >> Yeah. So we have an environment where reductor is um operating in our environment against all of our content. We have billions of pages of content. We have very big spurts of content when bakers are dropping in large data sets potentially up to a terabyte of data can get dropped in on a moment's notice. So we have an ingestion pipeline that takes all of that, passes it through reductive. We get that back. We we leverage turbopuffer as our vector database. So we're passing it off to turbopuffer. Um and then we build our graph rag on top of that. We're executing all of our graph rag queries uh using prop python code against all that. >> Yeah. Um I'm on the cell side of things. Um we're we are actually customers data >> every cell is [laughter] >> um curious about reductto. So we're sitting on a transaction like we have a a file system with a ton of transactions and we want to make that available to agents but there's a way of like obviously it's MNPI we want to anonymize that before it makes its way into something like >> we're going to either build this or buy this. So, it's very helpful conversation. That being said, giving it to the agents, you you mentioned before that you built a custom eval. And so, I I'm curious about how y'all went about

this eval potentially open sourced. Amazing. Um, but like curious about how you guys that eval. So our custom eval that's related to redu we have a few custom evals but the one that's most relevant to reductto is what we call VDRB bench. We haven't open source it. We're actually thinking about doing that thinking about partnering with open AAI and a paper around it. Um what we're doing with VDRBench is we basically take a common virtual data room of hundreds of documents. Um and we ask a few hundred questions against that data room with a known set of answers against those. Um we run that every time we make changes to to pretty much anything. um the scores on that we expect to be higher than any of the uh frontier labs directly. So if you were to for example drop that same set of files into cloud projects or open AI projects, we need to make sure that we're beating that. Um and a lot of the reason that we beat that our data science team benchmarks of our recall rates at you know improving 22% because of our reductive parsing. Um you know a lot of that because of the the way we pull out the content. It's actually not the other side of it. A lot of people think that sort of all the work needs to be done on the other side.

the answering side and most of the work actually is on the extraction side. You get that right and everything else can follow.