Aniket PujariEarly data technology involved multiple databases storing and retrieving static information, or “data at rest”. Modern businesses, instead, rely on interconnected systems, requiring seamless data flow across applications. “In today’s dynamic environment, companies need ‘data in motion’ that constantly flows between applications. Confluent bridges this gap,” Kreps told Mint in an interview during his visit to India on Wednesday. He was in Bengaluru to speak at Confluent’s user conference called ‘Current’.
Confluent enables “data in motion” through Apache Kafka—a popular open-source platform that was co-created by Kreps himself. Originally developed at professional networking platform LinkedIn, and now managed by the Apache Software Foundation, industries use Kafka for real-time operations—retailers track sales, banks manage transactions, and logistics firms coordinate deliveries.
AI applications, too, depend on real-time data integration, making streaming essential for leveraging language models in business operations. This shift has driven Confluent’s growth, positioning it at the core of real-time data streaming across industries, according to Kreps.
AI adoption: India leads the way
India, he believes, remains at the forefront of tech adoption, leading in both AI and real-time data streaming. Confluent has partnerships with companies like Jio Platforms, Swiggy, Meesho, and Viacom in India. “We’ve partnered with Jio Platforms, which offers a cloud service, and we’re making Confluent available on their Jio Cloud, part of the Azure West region,” he said.
For instance, Jio’s network and companies like Viacom are leveraging Confluent’s technology for analytics—scaling events like the Indian Premier League (IPL) opening day viewership to over 590 million. Swiggy, on its part, uses Confluent for data metrics, governance capabilities, and real-time delivery insights to help its data team streamline complex workflows and scale operations quickly during peak demand.
Kreps underscored that the Confluent Platform and Connect portfolio are “fully managed by engineering teams based in India—built in India for the world”. He added that India is one of Confluent’s “biggest and fastest-growing regions, with the team growing 50% last year”.
“Our engineering presence here is on par with our Bay Area team, driving major projects like Table Flow, a feature we launched today.”
Table Flow helps Confluent users turn their streaming data from Kafka into organized tables (in Apache Iceberg or Delta Lake). This makes it easier to store, manage, and analyze real-time data in a structured way so that businesses can quickly access and use the information for reporting, analytics, or AI and Generative AI (GenAI) applications.
Confluent plans to continue hiring in the Asia-Pacific region and grow it by another 20%, especially in India. Kreps added that India’s investment in digital infrastructure has positioned it as a global leader. “Payments here use Kafka and Confluent extensively. The scale is orders of magnitude higher than in the US, where payments are periodic. In India, they happen everywhere, all the time. It’s exciting to be part of this transformation,” Kreps said.
The missing piece: High-quality data
To be sure, with AI’s rapid advancement, integrating real-time data has never been more crucial. However, he acknowledged that AI adoption presents challenges, particularly for enterprises managing legacy systems and being presented with a multitude of application programming interfaces (APIs). “AI models trained on general internet knowledge lack company-specific insights—unlocking that data is key,” he explained.
Furthermore, AI development requires structured, high-quality data, something many businesses struggle with. “Many tech teams are still learning how to integrate AI effectively into workflows,” he added.
According to Kreps, who is also on the board of AI startup Anthropic, which develops Claude LLM models, there’s “just incredible work happening to make these AI models smarter and smarter, and that solves a lot of problems in terms of the scope and scale and reliability of what they can do”.
He acknowledged, though, that there are still “some big missing pieces” beyond the models. For instance, a lot of companies’ data “is spread over 1,000 different databases and applications and SaaS (software-as-a-service) systems”, which makes integration with these models a “big challenge”.
The good news, however, is that while there is “a lot of different models, and a lot of innovation on the models, in terms of the interface to customers, there’s a fair amount of similarity”. The key, Kreps believes, is to feed models live data from various databases and applications, which is particularly important for AI-driven agentic systems, where models act autonomously. Without accurate, up-to-date data, these systems struggle to make informed decisions.
As models advance, more AI agentic applications will become practical, reshaping enterprise workflows, according to Kreps.
Being the co-creator of Kafka, Kreps is naturally bullish on open source, too. “Open-source AI models have driven innovation, with companies like Meta and DeepSeek producing strong results. The impact on future models remains to be seen.”
Meanwhile, as AI transforms industries, its effect on jobs remains a topic of debate. “In the short term, AI boosts productivity without reducing employment. For example, coding tools enhance efficiency but don’t replace engineers,” Kreps noted.
However, long-term impacts are harder to predict. “While past innovations didn’t cause mass unemployment, they did disrupt industries. AI’s broad capabilities make its future impact more uncertain,” he said, adding that adaptation is, hence, essential. “Technology evolves rapidly, and staying still means getting left behind,” Kreps concluded.
