mirror of https://github.com/huggingface/text-generation-inference.git synced 2026-10-12 02:15:47 +00:00

Files

T

History

Nicolas Patry ddf0b02240 All the assertions.

Invariants added

Remove the logs.

2025-03-04 13:32:05 +01:00

custom_kernels

All integration tests back everywhere (too many failed CI). (#2428 )

2024-08-16 21:19:46 +02:00

exllama_kernels

Update ROCM libs and improvements (#2579 )

2024-09-30 10:54:32 +02:00

exllamav2_kernels

Update ROCM libs and improvements (#2579 )

2024-09-30 10:54:32 +02:00

tests

feat: improve star coder to support multi lora layers (#2883 )

2025-01-16 16:23:55 -05:00

text_generation_server

All the assertions.

2025-03-04 13:32:05 +01:00

.gitignore

Impl simple mamba model (#1480 )

2024-02-08 10:19:45 +01:00

bounds-from-nix.py

Sync (most) server dependencies with Nix (#2782 )

2024-12-03 04:04:06 +01:00

Makefile

Moving to uv instead of poetry. (#2919 )

2025-01-17 12:32:00 +01:00

Makefile-awq

chore: add pre-commit (#1569 )

2024-02-16 11:58:58 +01:00

Makefile-eetq

Sync (most) server dependencies with Nix (#2782 )

2024-12-03 04:04:06 +01:00

Makefile-exllamav2

Upgrading exl2. (#2415 )

2024-08-14 11:58:08 +02:00

Makefile-flash-att

Hotfixing make install. (#2008 )

2024-06-04 23:34:03 +02:00

Makefile-flash-att-v2

Add Flash decoding kernel ROCm (#2855 )

2025-01-13 11:12:35 +01:00

Makefile-flashinfer

Trying to put back the archlist (to fix the oom). (#2947 )

2025-01-24 09:32:17 +01:00

Makefile-lorax-punica

Enable multiple LoRa adapters (#2010 )

2024-06-25 14:46:27 -04:00

Makefile-selective-scan

chore: add pre-commit (#1569 )

2024-02-16 11:58:58 +01:00

Makefile-vllm

Update vllm kernels for ROCM (#2826 )

2024-12-18 12:44:42 +01:00

pyproject.toml

Prepare for release 3.1.0 (#2972 )

2025-01-31 14:19:01 +01:00

README.md

chore: add pre-commit (#1569 )

2024-02-16 11:58:58 +01:00

requirements_cuda.txt

Sync (most) server dependencies with Nix (#2782 )

2024-12-03 04:04:06 +01:00

requirements_intel.txt

Sync (most) server dependencies with Nix (#2782 )

2024-12-03 04:04:06 +01:00

requirements_rocm.txt

Sync (most) server dependencies with Nix (#2782 )

2024-12-03 04:04:06 +01:00

uv.lock

Update to moe-kernels 0.8.0 (#2966 )

2025-01-29 18:19:55 +01:00

README.md

Text Generation Inference Python gRPC Server

A Python gRPC server for Text Generation Inference

Install

make install

Run

make run-dev