Langchain js document loader. Installation The LangChain CSVLoader integration lives in the @langchain/community integration package. By default, one document will be created for each page in Figma Figma is a collaborative web application for interface design. It supports both the new syntax with options object and the legacy syntax for backward compatibility. By default, one document will be created for each chapter in the EPUB file, you can change this behavior by setting the splitChapters option to false. Only available on Node. DocumentLoaders load data into the standard LangChain Document format. Integrations You can find available integrations on the Document loaders integrations page. Documents and Document Loaders LangChain implements a Document abstraction, which is intended to represent a unit of text and associated metadata. docx format and the legacy . Available in both Python- and Javascript-based libraries, LangChain’s tools and APIs simplify the process of building LLM-driven applications like chatbots and AI agents. js introduction docs. When you use all LangChain products, you'll build better, get to production quicker, and grow visibility -- all with less set up and friction. js and browser environments, but a Chrome extension’s service worker runtime is neither. It provides essential building blocks like chains, agents, and memory components that enable developers to create sophisticated AI workflows beyond simple prompt-response interactions. LangChain implements a standard interface for large language models and related technologies, such as embedding models and vector stores, and integrates with hundreds of providers. The second argument is a JSONPointer to the property to extract from each JSON object in the file. Unstructured currently supports loading of text files, powerpoints, html, pdfs, images, and more. Document Loaders are usually used to load a lot of Documents in a single run. How to: load CSV data How to: load data from a directory How to: load PDF files How to: write a custom document loader How to: load HTML data How to: load Markdown data Text It represents a document loader that loads documents from a buffer. 2 days ago · LangChain is a powerful framework that simplifies the development of applications powered by large language models (LLMs). It returns an array of Document instances. Methods load load(): Promise<Document<Record<string, any>>[]> Extracts the text content from the loaded document using the selector and creates a Document instance with the extracted text and metadata. What Are Document Loaders? Document loaders are tools To handle different types of documents in a straightforward way, LangChain provides several document loader classes. You can run the loader in one of two modes: “single” and “elements”. jsload load(): Promise<Document[]> An asynchronous method that loads the web page. Class hierarchy: The DocxLoader allows you to extract text data from Microsoft Word documents. , FAISS, Pinecone). Each file will be passed to the matching loader, and the resulting documents will be concatenated together. jsReturns Promise<Document<Record<string, any>>[]> A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter. The metadata Documentation for LangChain. It extends the BaseDocumentLoader and implements the FigmaLoaderParams interface. This example goes over how to load data from folders with multiple files. Class hierarchy: Jun 23, 2023 · AsyncHtmlLoader loads raw HTML from a list of URLs concurrently. Discover how each tool fits into the LLM application stack and when to use them. It extends the BaseDocumentLoader class and implements the load() method. The BaseDocumentLoader class provides a few convenience methods for loading documents from a variety of sources. This integration provides Docling's capabilities via the DoclingLoader document loader. Hierarchy DocumentLoader Implemented by BaseDocumentLoader Defined in langchain-core/dist/document_loaders/base. Documentation for LangChain. Otherwise, it creates a new Document instance Only available on Node. What is LangChain? Before we dive into the specifics of LangChain Document Loaders, let's take a step back and understand what LangChain is. This example goes over how to load data from JSONLines or JSONL files. They Use document loaders to load data from a source as Document 's. Loader features When loading content from a website, we may want to process load all URLs on a page. A Document is a piece of text and associated metadata. It represents a document loader that loads documents from a buffer. May 31, 2025 · In this guide, we’ll explore what document loaders are, how they work, and how to use them in real-world projects. Here we demonstrate: How to load from a filesystem, including use of wildcard patterns; How to use multithreading for file I/O; How to use custom loader classes to parse specific file types (e. js libraries to fetch the transcript and video metadata. PDF Portable Document Format (PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems. Subclassing BaseDocumentLoader You can extend the BaseDocumentLoader class directly. It uses the getDocument function from the PDF. How to load HTML The HyperText Markup Language or HTML is the standard markup language for documents designed to be displayed in a web browser. jsA method that takes a raw buffer and metadata as parameters and returns a promise that resolves to an array of Document instances. How to create a custom Document Loader Overview Applications based on LLMs frequently entail extracting data from databases or files, like PDFs, and converting it into a format that LLMs can utilize. Framework to build resilient language agents as graphs. For detailed documentation of all JSONLoader features and configurations head to the API reference. Setup To access WebPDFLoader document loader you’ll need to install the @langchain/community integration, along with the pdf-parse package: Credentials If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below: Custom document loaders If you want to implement your own Document Loader, you have a few options. This also A class that extends the BufferLoader class. It supports both the modern . 📄️ Vsdx A visio file (with extension . Sep 15, 2024 · Step 6: Leveraging LangChain Utilities LangChain provides various utilities and functionalities that can be leveraged once the HTML document is loaded and processed. html. Using PyPDF Load PDF using pypdf into array of documents Docling Docling parses PDF, DOCX, PPTX, HTML, and other formats into a rich unified representation including document layout, tables etc. File Loaders Compatibility Only available on Node. Here we cover how to load Markdown documents into LangChain Document objects that we can use downstream. They do not involve the local file system. It provides a standard interface for chains, many integrations with other tools, and end-to-end chains for common applications. 4 days ago · Learn the key differences between LangChain, LangGraph, and LangSmith. 3 days ago · Learn how to use the LangChain ecosystem to build, test, deploy, monitor, and visualize complex agentic workflows. Example folder: Webpages, with Playwright Compatibility Only available on Node. LLM Integration: Supplies retrieved content as context. Jun 2, 2025 · In this guide, we’ll explore what document loaders are, how they work, and how to use them in real-world projects. It also integrates with multiple AI models like Google's Gemini and OpenAI for generating insights from the loaded documents. Below we How to load Markdown Markdown is a lightweight markup language for creating formatted text using a plain-text editor. Document loaders load data into LangChain's expected format for use-cases such as retrieval-augmented generation (RAG). Setup To run this loader you will need to create Documentation for LangChain. For example, there are document loaders for loading a simple . How to: load PDF files How to: load web pages How to: load CSV data How to: load data from a directory How to: load HTML data How to: load JSON data How to: load Markdown data How to: load Microsoft Office data How to: write a custom document loader A document loader that loads documents from a directory. It represents a document loader that loads documents from PDF files. They facilitate the seamless integration and processing of diverse data sources, such as YouTube, Wikipedia, and GitHub, into Document objects. This notebook covers how to load data from the Figma REST API into a format that can be ingested into LangChain, along with example usage for code generation. To access UnstructuredLoader document loader you’ll need to install the @langchain/community integration package, and create an Unstructured account and get an API key. Parsing HTML files often requires specialized tools. This notebook covers how to use Unstructured document loader to load files of many types. By default, one document will be created for each page in the PDF file, you can change this behavior by setting the splitPages option to false. Web pages contain text, images, and other multimedia elements, and are typically represented with HTML. , making them ready for generative AI workflows like RAG. A document loader that uses the Unstructured API to load unstructured documents. It uses the youtube-transcript and youtubei. jsInterface that defines the methods for loading and splitting documents. This example goes over how to load data from a GitHub repository. ts:6 Index Sep 15, 2024 · Step 6: Leveraging LangChain Utilities LangChain provides various utilities and functionalities that can be leveraged once the HTML document is loaded and processed. In LangChain, this usually involves creating Document objects, which encapsulate the extracted text (page_content) along with metadata—a dictionary containing details about the document, such as document_loaders # Document Loaders are classes to load Documents. document_loaders # Document Loaders are classes to load Documents. Jun 29, 2023 · In this comprehensive guide, we'll unravel the mysteries of LangChain Document Loaders and show you how they can be a game-changer in your language model applications. One document will be created for each JSON object in the file. UnstructuredHTMLLoader ¶ class langchain_community. This can be found in the url when navigating to your space e. Document loaders Document Loaders are responsible for loading documents from a variety of sources. Use document loaders to load data from a source as Document 's. document_loaders. , CSV, PDF, HTML) into standardized Document objects for LLM applications. What Are Document Loaders? Document loaders are tools that help you bring external content into your LangChain application in a structured way. Example files: Jun 2, 2025 · Let’s put document loaders to work with a real example using LangChain. g. These loaders are used to load web resources. Embeddings: Convert documents to semantic vectors. Follow their code on GitHub. https How to load documents from a directory LangChain's DirectoryLoader implements functionality for reading files from disk into LangChain Document objects. Playwright is a Node. This project demonstrates the use of LangChain's document loaders to process various types of data, including text files, PDFs, CSVs, and web pages. txt file, for loading the text contents of any web page, or even for loading a transcript of a YouTube video. Each file will be passed to the matching loader This covers how to use WebBaseLoader to load all text from HTML webpages into a document format that we can use downstream. It has three attributes: pageContent: a string representing the content; metadata: records of arbitrary metadata; id: (optional) a string identifier for the document. Returns Promise<Document[]> A Promise that resolves to an array of Document instances This example goes over how to load data from EPUB files. This example covers how to load HTML documents from a list of URLs into the Document format that we can use downstream. Depending on the file type, additional dependencies are required. How to: load CSV data How to: load data from a directory How to: load PDF files How to: write a custom document loader How to: load HTML data How to: load Markdown data Text This covers how to use WebBaseLoader to load all text from HTML webpages into a document format that we can use downstream. A class that extends the BufferLoader class. Jul 9, 2025 · The startup, which sources say is raising at a $1. We will cover: Basic usage; Parsing of Markdown into elements such as titles, list items, and text. , code); How to handle errors, such as those due Returns Promise<Document<Record<string, any>>[]> A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter. jsClass representing a document loader for loading Figma files. The load () method is left abstract and needs to be implemented by subclasses. Document loaders are designed to load document objects. d. For example, let’s look at the LangChain. For detailed documentation of all DirectoryLoader features and configurations head to the API reference. It then iterates over each page of the PDF, retrieves the text content using the getTextContent method, and joins the text items to form the page This notebook provides a quick overview for getting started with DirectoryLoader document loaders. . How to write a custom document loader If you want to implement your own Document Loader, you have a few options. The metadata Apr 2, 2024 · LangChain provides document loaders that run in Node. This covers how to load PDF documents into the Document format that we use downstream. LangChain is an open source orchestration framework for application development using large language models (LLMs). For more custom logic for loading webpages look at some child class examples such as IMSDbLoader, AZLyricsLoader, and CollegeConfidentialLoader. It A document loader for loading data from YouTube videos. The load () method sends a partitioning request to the Unstructured API and retrieves the partitioned elements. It represents a document loader that loads documents from DOCX files. LangChain has hundreds of integrations with various data sources to load data from: Slack, Notion, Google Drive, etc. Interface Documents loaders implement the BaseLoader interface. How to: parse XML output How to: try to fix errors in output parsing Document loaders Document Loaders are responsible for loading documents from a variety of sources. This covers how to load document objects from an audio file using the Open AI Whisper API. Jul 15, 2024 · LangChain Document Loaders convert data from various formats (e. They may include links to other pages or resources. It uses the parseOfficeAsync function from the officeparser module to extract the raw text content from the buffer. jsA method that loads the text file or blob and returns a promise that resolves to an array of Document instances. LangChain has 208 repositories available. This covers how to load document objects from pages in a Confluence space. This covers how to load HTML documents into a LangChain Document objects that we can use downstream. js library that provides a high-level API for controlling multiple browser engines, including Chromium, Firefox, and WebKit. Overview The presented DoclingLoader component enables you to: use various document types in your LLM A class that extends the BufferLoader class. LangChain's products work seamlessly together to provide an integrated solution for every step of the application development journey. Sitemap Loader This notebook goes over how to use the SitemapLoader class to load sitemaps into Document s. Multiple individual files This example goes over how to load data from multiple file paths. This notebook provides a quick overview for getting started with JSON document loader. One document will be created for each webpage. LangChain implements an UnstructuredLoader class. The constructor takes a config object as a parameter, which contains the access token, an array of node IDs, and the file key. If the extracted powerpoint content is empty, it returns an empty array. For example, let's look at the LangChain. Returns Promise<Document<Record<string, any>>[]> A Promise that resolves to an array of Document Recursive URL Loader When loading content from a website, we may want to process load all URLs on a page. UnstructuredHTMLLoader(file_path: Union[str, List[str], Path, List[Path]], *, mode: str = 'single', **unstructured_kwargs: Any) [source] ¶ Load HTML files using Unstructured. The load() method is implemented to read the text from the file or blob, parse it using the parse() method, and create a Document instance for each parsed page. This example goes over how to load data from PDF files. It has a constructor that takes a filePathOrBlob parameter representing the path to the word file or a Blob object, and an optional options parameter of type DocxLoaderOptions 📄️ URL This example covers how to load HTML documents from a list of URLs into the Document format that we can use downstream. js. Otherwise, it calls the loadResults () method to load the results from the web page. Credentials You'll need to set up an access token and provide it along with your confluence username in order to authenticate the request You'll also need the space key for the space containing the pages to load as documents. Document loaders provide a "load" method for loading data as documents from a configured source. LangChain integrates with a host of parsers that are appropriate for web pages. Retriever: Finds relevant docs for a query. The load() method is implemented to read the buffer contents and metadata based on the type of filePathOrBlob, and then calls the parse() method to parse the buffer and return the documents. LangChain is a software framework that helps facilitate the integration of large language models (LLMs) into applications. This has many interesting child pages that we may want to load, split, and later retrieve in bulk. Class that extends the BaseDocumentLoader class and implements the DocumentLoader interface. Let’s dive in. The challenge is traversing the tree of child pages and assembling a list! We do this using the RecursiveUrlLoader. If the webPath includes "item", it calls the loadComments () method to load the comments from the web page. Setup First, we need to install the langchain package: Document loaders are designed to load document objects. If you use “single Setup To access CSVLoader document loader you’ll need to install the @langchain/community integration, along with the d3-dsv@2 peer dependency. Returns Promise<Document<Record<string, any>>[]> A Promise that resolves to an array of Document This covers how to load audio (and video) transcripts as document objects from a file using the AssemblyAI API. It reads the text from the file or blob using the readFile function from the node:fs/promises module or the text() method of the blob. Jul 23, 2025 · LangChain is an open-source framework designed to simplify the creation of applications using large language models (LLMs). You can optionally provide a s3Config parameter to specify your bucket region, access key, and secret access key. It stores information about the structure, layout, and graphical elements of a diagram. As a language model integration framework, LangChain's use-cases largely overlap with those of language models in general, including document analysis and summarization, chatbots, and code analysis. 1 billion valuation, helps developers at companies like Klarna and Rippling use off-the-shelf AI models to create new applications. It represents a document loader for scraping web pages using Puppeteer. jsAbstract class that provides a default implementation for the loadAndSplit () method from the DocumentLoader interface. vsdx) is associated with Microsoft Visio, a diagram creation software. This example goes over how to load data from webpages using Playwright. It then parses the text using the parse() method and creates a Document instance for each parsed page. The challenge is traversing the tree of child pages and assembling a list! How to load data from a directory This covers how to load all documents in a directory. Dec 9, 2024 · langchain_community. The second argument is a map of file extensions to loader factories. js library to load the PDF from the buffer. doc format. The right parser will depend on your needs. These loaders are used to load files given a filesystem path or a Blob object. interface DocumentLoader { load (): Promise<Document<Record<string, any>>[]>; loadAndSplit (textSplitter?: BaseDocumentTransformer<DocumentInterface<Record<string Once Unstructured is configured, you can use the S3 loader to load files and then convert them into a Document. It creates a Document instance for each element and returns an array of Document instances. This guide covers how to load web pages into the LangChain Document format that we use downstream. Jul 23, 2025 · Retrieval-Augmented Generation (RAG) Components: Document loaders: Ingest data from HTML, DOC, S3, etc. Say you have a PDF you’d like to load into your app; maybe a research paper, product guide, or internal policy doc. Vector database: Store vectors for similarity search (e. It represents a document loader that loads documents from a text file. umyycv pnud xwkjq nzhzyox jyc rjl hbzn uid kbrysrq vdam