[GH-ISSUE #2216] [BUG]: Folders are not being imported when using Github data connector #1448

Closed
opened 2026-02-22 18:24:51 -05:00 by yindo · 3 comments
Owner

Originally created by @Archisman-Mridha on GitHub (Sep 4, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2216

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

I was trying to import this repo : https://github.com/tikv/agatedb, using the github data connector. It just imported the 7 files at the top level and not the folders.

Are there known steps to reproduce?

No response

Originally created by @Archisman-Mridha on GitHub (Sep 4, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2216 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? I was trying to import this repo : https://github.com/tikv/agatedb, using the github data connector. It just imported the 7 files at the top level and not the folders. ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:24:51 -05:00
yindo closed this issue 2026-02-22 18:24:51 -05:00
Author
Owner

@timothycarambat commented on GitHub (Sep 4, 2024):

It says on the UI why you get that limitation.
Screenshot 2024-09-04 at 12 43 24 PM

@timothycarambat commented on GitHub (Sep 4, 2024): It says on the UI why you get that limitation. <img width="629" alt="Screenshot 2024-09-04 at 12 43 24 PM" src="https://github.com/user-attachments/assets/bfd6c318-ae2e-4c03-95c1-bda001f7bf0e">
Author
Owner

@Archisman-Mridha commented on GitHub (Sep 6, 2024):

I did provide the Github Access Token! @timothycarambat

@Archisman-Mridha commented on GitHub (Sep 6, 2024): I did provide the Github Access Token! @timothycarambat
Author
Owner

@timothycarambat commented on GitHub (Sep 6, 2024):

Still unable to replicate:

  • Create PAT (Fine-grained), do not change any additional options
  • Obtain key
  • Open connector and use https://github.com/tikv/agatedb as repo URL
  • Paste in PAT from earlier step
  • Confirm master branch selection
  • 67 files scraped, does include nested
[Github Loader]: Saving .github/workflows/bench.yml to tikv-agatedb-master-f0db
[Github Loader]: Saving .github/workflows/test.yml to tikv-agatedb-master-f0db
[Github Loader]: Saving .github/workflows/tsan to tikv-agatedb-master-f0db
[Github Loader]: Saving .gitignore to tikv-agatedb-master-f0db
[Github Loader]: Saving Cargo.toml to tikv-agatedb-master-f0db
[Github Loader]: Saving LICENSE to tikv-agatedb-master-f0db
[Github Loader]: Saving Makefile to tikv-agatedb-master-f0db
[Github Loader]: Saving README.md to tikv-agatedb-master-f0db
[Github Loader]: Saving agate_bench/Cargo.toml to tikv-agatedb-master-f0db
[Github Loader]: Saving agate_bench/src/main.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving benches/bench_common.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving benches/bench_iterator.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving benches/bench_table.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving benches/benches_agate_rocks.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving benches/common.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving codecov.yml to tikv-agatedb-master-f0db
[Github Loader]: Saving proto/Cargo.toml to tikv-agatedb-master-f0db
[Github Loader]: Saving proto/build.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving proto/src/lib.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving proto/src/proto/meta.proto to tikv-agatedb-master-f0db
[Github Loader]: Saving proto/src/proto/rustproto.proto to tikv-agatedb-master-f0db
[Github Loader]: Saving rustfmt.toml to tikv-agatedb-master-f0db
[Github Loader]: Saving scripts/gen.cmd to tikv-agatedb-master-f0db
[Github Loader]: Saving skiplist/Cargo.toml to tikv-agatedb-master-f0db
[Github Loader]: Saving skiplist/benches/bench.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving skiplist/src/arena.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving skiplist/src/key.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving skiplist/src/lib.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving skiplist/src/list.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving skiplist/tests/tests.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/batch.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/bloom.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/checksum.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/closer.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/db.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/db/opt.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/db/tests.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/entry.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/error.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/format.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/iterator.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/iterator_trait.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/levels.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/levels/compaction.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/levels/handler.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/levels/tests.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/lib.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/managed_db.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/manifest.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/memtable.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/ops.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/ops/oracle.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/ops/snapshot.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/ops/transaction.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/ops/transaction_test.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/opt.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/table.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/table/builder.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/table/concat_iterator.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/table/iterator.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/table/merge_iterator.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/table/tests.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/util.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/value.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/value_log.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/wal.rs to tikv-agatedb-master-f0db
[Github Loader]: Saving src/watermark.rs to tikv-agatedb-master-f0db
@timothycarambat commented on GitHub (Sep 6, 2024): Still unable to replicate: - Create PAT (Fine-grained), do not change any additional options - Obtain key - Open connector and use `https://github.com/tikv/agatedb` as repo URL - Paste in PAT from earlier step - Confirm `master` branch selection - 67 files scraped, does include nested ``` [Github Loader]: Saving .github/workflows/bench.yml to tikv-agatedb-master-f0db [Github Loader]: Saving .github/workflows/test.yml to tikv-agatedb-master-f0db [Github Loader]: Saving .github/workflows/tsan to tikv-agatedb-master-f0db [Github Loader]: Saving .gitignore to tikv-agatedb-master-f0db [Github Loader]: Saving Cargo.toml to tikv-agatedb-master-f0db [Github Loader]: Saving LICENSE to tikv-agatedb-master-f0db [Github Loader]: Saving Makefile to tikv-agatedb-master-f0db [Github Loader]: Saving README.md to tikv-agatedb-master-f0db [Github Loader]: Saving agate_bench/Cargo.toml to tikv-agatedb-master-f0db [Github Loader]: Saving agate_bench/src/main.rs to tikv-agatedb-master-f0db [Github Loader]: Saving benches/bench_common.rs to tikv-agatedb-master-f0db [Github Loader]: Saving benches/bench_iterator.rs to tikv-agatedb-master-f0db [Github Loader]: Saving benches/bench_table.rs to tikv-agatedb-master-f0db [Github Loader]: Saving benches/benches_agate_rocks.rs to tikv-agatedb-master-f0db [Github Loader]: Saving benches/common.rs to tikv-agatedb-master-f0db [Github Loader]: Saving codecov.yml to tikv-agatedb-master-f0db [Github Loader]: Saving proto/Cargo.toml to tikv-agatedb-master-f0db [Github Loader]: Saving proto/build.rs to tikv-agatedb-master-f0db [Github Loader]: Saving proto/src/lib.rs to tikv-agatedb-master-f0db [Github Loader]: Saving proto/src/proto/meta.proto to tikv-agatedb-master-f0db [Github Loader]: Saving proto/src/proto/rustproto.proto to tikv-agatedb-master-f0db [Github Loader]: Saving rustfmt.toml to tikv-agatedb-master-f0db [Github Loader]: Saving scripts/gen.cmd to tikv-agatedb-master-f0db [Github Loader]: Saving skiplist/Cargo.toml to tikv-agatedb-master-f0db [Github Loader]: Saving skiplist/benches/bench.rs to tikv-agatedb-master-f0db [Github Loader]: Saving skiplist/src/arena.rs to tikv-agatedb-master-f0db [Github Loader]: Saving skiplist/src/key.rs to tikv-agatedb-master-f0db [Github Loader]: Saving skiplist/src/lib.rs to tikv-agatedb-master-f0db [Github Loader]: Saving skiplist/src/list.rs to tikv-agatedb-master-f0db [Github Loader]: Saving skiplist/tests/tests.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/batch.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/bloom.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/checksum.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/closer.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/db.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/db/opt.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/db/tests.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/entry.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/error.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/format.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/iterator.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/iterator_trait.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/levels.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/levels/compaction.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/levels/handler.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/levels/tests.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/lib.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/managed_db.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/manifest.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/memtable.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/ops.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/ops/oracle.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/ops/snapshot.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/ops/transaction.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/ops/transaction_test.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/opt.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/table.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/table/builder.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/table/concat_iterator.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/table/iterator.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/table/merge_iterator.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/table/tests.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/util.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/value.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/value_log.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/wal.rs to tikv-agatedb-master-f0db [Github Loader]: Saving src/watermark.rs to tikv-agatedb-master-f0db ```
yindo changed title from [BUG]: Folders are not being imported when using Github data connector to [GH-ISSUE #2216] [BUG]: Folders are not being imported when using Github data connector 2026-06-05 14:40:49 -04:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1448