LLM
Scalable PDF Document Processing with DataChain and Unstructured.io
Extract and parse text from documents and create vector embeddings in a scalable and distributed way (and less than 70 lines of code).
Enforcing JSON Outputs in Commercial LLMs
The results of our tests on the structured outputs of Google Gemini Pro, Anthropic Claude, and OpenAI GPT. DataChain used for evaluation.
Announcing DataChain
Introducing DataChain - a new open-source tool to curate and process unstructured data using local ML models, and LLM calls.
Leveraging LLMs in Chatbots: The DVC Approach
Read how DVC can optimize the development process for chatbots built on Large Language Models.
Fine-Tuning Large Language Models with a Production-Grade Pipeline
This post describes a production ML pipeline for fine-tuning large language models using DVC, SkyPilot, HuggingFace Transformers, and quantization techniques.
