Cloudera and Pinecone form strategic partnership

Cloudera and Pinecone have formed a strategic partnership that integrates Pinecone’s AI vector database expertise into Cloudera’s open data platform, aimed at transforming the way organizations harness the power of AI to streamline operations and improve customer experiences.

A market leader, Pinecone’s vector database is critical infrastructure for Generative AI. Pinecone is optimized to store AI representations of data (vector embeddings) and search through them by semantic similarity, something traditional databases are very inefficient at doing. This capability is necessary for adding context to queries against applications that use Large Language Models (LLMs). That added context significantly cuts down on erroneous outputs – often referred to as "hallucinations" – helping search and Generative AI applications deliver responses that are accurate and relevant.

The partnership will see Cloudera integrate Pinecone’s best-in-class vector database into Cloudera Data Platform (CDP), enabling organizations to more easily build and deploy highly scalable, real-time, AI-powered applications on Cloudera. This includes the release of a new Applied ML Prototype (AMP) that will allow developers to more quickly create and augment new knowledge bases from data on their own website, as well as pre-built connectors that will enable customers to more quickly set up ingest pipelines in AI applications. In the AMP, Pinceone’s vector database uses these knowledge bases to imbue context into chatbot responses, helping to ensure useful outputs.

Customers can use this same architecture to set up or improve support chatbots or internal support search systems. This enables them to reduce operational costs by decreasing expensive human case-handling efforts and improving the customer experience with faster resolution times. More information on this AMP and how vector databases add context to AI applications can be found in our blog post here.

"Cloudera's extensive expertise in data management combined with Pinecone's cutting-edge vector database creates a formidable partnership. A lot of our customers already manage their data with Cloudera. Now it will be easier than ever for them to build AI applications using their embeddings stored with us and data stored with Cloudera. Together we will enable organizations to deliver unparalleled personalized experiences, drive user engagement, and achieve business success," Elan Dekel, Vice President of Product, Pinecone.

"We are excited to bring the power of Pinecone vector database and semantic search capabilities to our public cloud customers to accelerate generative AI use cases, and significantly improve the developer experience at scale." Abhas Ricky, Chief Strategy Officer, Cloudera.

“Integration of Pinecone with CDP adds a very critical new functionality that will help clients build generative AI applications,” said Sanjeev Mohan, founder of SanjMo and former Gartner analyst. “In addition, the planned integration between the open source Apache NiFi-based Cloudera Data Flow (CDF) and Pinecone further bolsters CDP’s emphasis on universal data distribution for AI. CDP customers can bring AI to where their data resides - on-premises, in the cloud or on the edge.”