Ep 43: CEO/Co-Founder of Contextual AI Douwe Kiela Reaction to o1, What’s Next in Reasoning and Innovations in Post-Training

"I think it's very encouraging to see that this works so well for reasoning, specifically. My own team has been working on very similar ideas, but they're much more on the retrieval side of things."

"Now that we have a model out there like this, I mean, should we expect most future models to follow this approach? Not necessarily. I think it depends a little bit on the latency constraints of the deployment."

"So I really think it's very important to take a systems approach rather than just a model approach, as the model is just one part of a much larger system that must be built around it to solve the problem effectively."

"Understanding sort of the risk of a deployment and the real accuracy of the system is hugely important. I would love there to be kind of a standard way to do that."

"It's very hard to contextualize language models properly. You have to encode the query basically twice – once for the retrieval step and once for the generation step."

"The reality is that extracting some information from a PDF is very, very hard. And there just isn't anything good out there in the open."

"I think a lot of people are not taking evaluation seriously enough. They just have like a spreadsheet somewhere with like 50 examples and then the super high variance. It's just, it's not principled enough."

"We should have something very different for AI evaluations today compared to the traditional methods. Developers need to focus on what AI really needs, as their roles have shifted from deep technical knowledge to building integrations."

"Synthetic data doesn't really work or like you can't train a language model on its own data. So that paper is full of flaws. It's not actually a realistic scenario. This is again like a sort of PR flag plant."

"He was recently on this other podcast, Acquired, and it's really visionary leadership that he's doing there. Why give away LLAMA? What do you think is behind that?"

"I also think it does them great benefit. It's helping them a lot with hiring. It's really putting them as one of the top players in the field. Yeah. So they're one of the cool kids again."

"If the new platform is language models or generative AI, it is very important for him to not have to serve that up on somebody else's platform."

"Our company mission is to change the way the world works, like through AI. And yeah, if we can really make that better for everybody, I think that is a very noble mission and will have huge impact."

"I think we're just really at this amazing point in history where we get to see the world changing in front of our eyes, changing how the world works. So yeah, exciting times ahead for everyone."

"In most alternative futures, I would still end up working on Contextual because I think like work is the most obvious place where this technology is going to change everything, right?"

What is "Ep 43: CEO/Co-Founder of Contextual AI Douwe Kiela Reaction to o1, What’s Next in Reasoning and Innovations in Post-Training" about?

In Episode 43 of the Redpoint Ventures podcast, Douwe Kiela, CEO of Contextual AI, discusses his groundbreaking work in AI, including his pioneering paper on retrieval-augmented generation (RAG) and raising substantial funding to help create customizable language models for enterprises. Douwe reflects on innovations in AI, particularly regarding OpenAI's O1 model and its implications for future enterprise applications. He emphasizes the need for a systems-thinking approach in deploying language models and highlights the importance of accurately defining use cases and understanding customer feedback. The episode also explores the critical role of academic collaboration and the trend towards smaller AI models for edge devices. Additionally, it addresses ongoing challenges in the evaluation of AI systems and the transformative potential of generative AI across various sectors.

What are the key takeaways from "Ep 43: CEO/Co-Founder of Contextual AI Douwe Kiela Reaction to o1, What’s Next in Reasoning and Innovations in Post-Training"?

Douwe Kiela is a leading figure in AI, having authored the first paper on retrieval-augmented generation and raising significant funds to support contextual language models. A systems-thinking approach is essential for effectively deploying AI technologies, focusing on comprehensive solutions rather than just model performance. Understanding customer needs and use cases is crucial for successful AI applications, as many organizations struggle to articulate their goals.

Which podcast is this episode from?

"Ep 43: CEO/Co-Founder of Contextual AI Douwe Kiela Reaction to o1, What’s Next in Reasoning and Innovations in Post-Training" is an episode from Unsupervised Learning.