DirectoryLoader

Compatibility

Only available on Node.js.

This notebook provides a quick overview for getting started with DirectoryLoader document loaders. For detailed documentation of all DirectoryLoader features and configurations head to the API reference.

This example goes over how to load data from folders with multiple files. The second argument is a map of file extensions to loader factories. Each file will be passed to the matching loader, and the resulting documents will be concatenated together.

Example folder:

src/document_loaders/example_data/example/
├── example.json
├── example.jsonl
├── example.txt
└── example.csv

Overview

Integration details

Class	Package	Compatibility	Local	PY support
DirectoryLoader	@langchain/community	Node-only	✅	✅

Setup

To access DirectoryLoader document loader you’ll need to install the langchain package.

Installation

The LangChain DirectoryLoader integration lives in the langchain package:

tip

See this section for general instructions on installing integration packages.

npm

npm i langchain

yarn

yarn add langchain

pnpm

pnpm add langchain

Instantiation

Now we can instantiate our model object and load documents:

import { DirectoryLoader } from "langchain/document_loaders/fs/directory";
import {
  JSONLoader,
  JSONLinesLoader,
} from "langchain/document_loaders/fs/json";
import { TextLoader } from "langchain/document_loaders/fs/text";
import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";

const loader = new DirectoryLoader(
  "../../../../../../examples/src/document_loaders/example_data",
  {
    ".json": (path) => new JSONLoader(path, "/texts"),
    ".jsonl": (path) => new JSONLinesLoader(path, "/html"),
    ".txt": (path) => new TextLoader(path),
    ".csv": (path) => new CSVLoader(path, "text"),
  }
);

Load

const docs = await loader.load();
docs[0];

Unknown file type: Star_Wars_The_Clone_Wars_S06E07_Crisis_at_the_Heart.srt
Unknown file type: bitcoin.pdf
Unknown file type: notion.md
Unknown file type: bad_frontmatter.md
Unknown file type: frontmatter.md
Unknown file type: no_frontmatter.md
Unknown file type: no_metadata.md
Unknown file type: tags_and_frontmatter.md
Unknown file type: test.mp3

Document {
  pageContent: 'Foo\nBar\nBaz\n\n',
  metadata: {
    source: '/Users/bracesproul/code/lang-chain-ai/langchainjs/examples/src/document_loaders/example_data/example.txt'
  },
  id: undefined
}

console.log(docs[0].metadata);

{
  source: '/Users/bracesproul/code/lang-chain-ai/langchainjs/examples/src/document_loaders/example_data/example.txt'
}

API reference

For detailed documentation of all DirectoryLoader features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_document_loaders_fs_directory.DirectoryLoader.html