View vllm on Snyk Open Source Advisor

Package Health Score

75 / 100

security
Security issues found
popularity
Popular
maintenance
Healthy
community
Active

Explore Similar Packages

Popularity

Popular

GitHub Stars: 29.62K
Forks: 4.47K
Contributors: 460

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package vllm, we found that it has been starred 29,617 times.

Security

Security issues found

Security and license risk for latest version

Release Date: Oct 17, 2024
Direct Vulnerabilities: 0
C
0
H
1
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
0
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

Apache-2.0

Security Policy

Yes

Snyk detected that the latest version of vllm has a security vulnerability.

We highly advise you to review these security issues.

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Fix it in your project with Snyk!

Maintenance

Healthy

Commit Frequency

Open Issues: 1.81K
Open PR: 444
Last Release: 22 days ago
Last Commit: 3 days ago

Further analysis of the maintenance status of vllm based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Healthy.

We found that vllm demonstrates a positive version release cadence with at least one new version released in the past 3 months.

As a healthy sign for on-going project maintenance, we found that the GitHub repository had at least 1 pull request or issue interacted with by the community.

Community

Active

Readme: Yes
Contributing.md: Yes
Code of Conduct: Yes
Contributors: 460
Funding: Yes

A good and healthy external contribution signal for vllm project, which invites more than one hundred open source maintainers to collaborate on the repository.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: >=3.8

Age: 1 year
Latest Release: 22 days ago
Dependencies: 14 Direct
Versions: 42
Maintainers: 4
Wheels: Linux

Readme

Easy, fast, and cheap LLM serving for everyone

Latest News 🔥

[2024/10] We have just created a developer slack (slack.vllm.ai) focusing on coordinating contributions and discussing features. Please feel free to join us there!
[2024/10] Ray Summit 2024 held a special track for vLLM! Please find the opening talk slides from the vLLM team here. Learn more from the talks from other vLLM contributors and users!
[2024/09] We hosted the sixth vLLM meetup with NVIDIA! Please find the meetup slides here.
[2024/07] We hosted the fifth vLLM meetup with AWS! Please find the meetup slides here.
[2024/07] In partnership with Meta, vLLM officially supports Llama 3.1 with FP8 quantization and pipeline parallelism! Please check out our blog post here.
[2024/06] We hosted the fourth vLLM meetup with Cloudflare and BentoML! Please find the meetup slides here.
[2024/04] We hosted the third vLLM meetup with Roblox! Please find the meetup slides here.
[2024/01] We hosted the second vLLM meetup with IBM! Please find the meetup slides here.
[2023/10] We hosted the first vLLM meetup with a16z! Please find the meetup slides here.
[2023/08] We would like to express our sincere gratitude to Andreessen Horowitz (a16z) for providing a generous grant to support the open-source development and research of vLLM.
[2023/06] We officially released vLLM! FastChat-vLLM integration has powered LMSYS Vicuna and Chatbot Arena since mid-April. Check out our blog post.

About

vLLM is a fast and easy-to-use library for LLM inference and serving.

vLLM is fast with:

State-of-the-art serving throughput
Efficient management of attention key and value memory with PagedAttention
Continuous batching of incoming requests
Fast model execution with CUDA/HIP graph
Quantizations: GPTQ, AWQ, INT4, INT8, and FP8.
Optimized CUDA kernels, including integration with FlashAttention and FlashInfer.
Speculative decoding
Chunked prefill

Performance benchmark: We include a performance benchmark at the end of our blog post. It compares the performance of vLLM against other LLM serving engines (TensorRT-LLM, SGLang and LMDeploy). The implementation is under nightly-benchmarks folder and you can reproduce this benchmark using our one-click runnable script.

vLLM is flexible and easy to use with:

Seamless integration with popular Hugging Face models
High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more
Tensor parallelism and pipeline parallelism support for distributed inference
Streaming outputs
OpenAI-compatible API server
Support NVIDIA GPUs, AMD CPUs and GPUs, Intel CPUs and GPUs, PowerPC CPUs, TPU, and AWS Neuron.
Prefix caching support
Multi-lora support

vLLM seamlessly supports most popular open-source models on HuggingFace, including:

Transformer-like LLMs (e.g., Llama)
Mixture-of-Expert LLMs (e.g., Mixtral)
Embedding Models (e.g. E5-Mistral)
Multi-modal LLMs (e.g., LLaVA)

Find the full list of supported models here.

Getting Started

Install vLLM with pip or from source:

pip install vllm

Visit our documentation to learn more.

Contributing

We welcome and value any contributions and collaborations. Please check out CONTRIBUTING.md for how to get involved.

Citation

If you use vLLM for your research, please cite our paper:

@inproceedings{kwon2023efficient,
  title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
  author={Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica},
  booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles},
  year={2023}
}

Contact Us

For technical questions and feature requests, please use Github issues or discussions.
For discussing with fellow users, please use Discord.
For coordinating contributions and development, please use Slack.
For security disclosures, please use Github's security advisory feature.
For collaborations and partnerships, please contact us at vllm-questions AT lists.berkeley.edu.

vllm dependencies

aiohttp einops importlib-metadata msgspec nvidia-ml-py partial-json-parser pillow protobuf psutil py-cpuinfo pyyaml pyzmq sentencepiece tqdm

FAQs

What is vllm?

A high-throughput and memory-efficient inference and serving engine for LLMs. Visit Snyk Advisor to see a full health score report for vllm, including popularity, security, maintenance & community analysis.

Is vllm popular?

The python package vllm receives a total of 76,475 weekly downloads. As such, vllm popularity was classified as a popular. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is vllm well maintained?

We found that vllm demonstrated a healthy version release cadence and project activity. It has a community of 460 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is vllm safe to use?

Security issues were found while scanning the latest version of vllm, and a total of 0 vulnerabilities were detected. It is highly advised to conduct a security review before using this package. View the full security scan results.

Last updated on 7 November-2024, at 13:53 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs For more information about how to use this package see README

Ensure you're using the healthiest python packages

Package Health Score

Explore Similar Packages

Popularity

Direct Usage Popularity

Security

Security and license risk for latest version

Maintenance

Commit Frequency

Community

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Package

Readme

Easy, fast, and cheap LLM serving for everyone

About

Getting Started

Contributing

Sponsors

Citation

Contact Us

vllm dependencies

FAQs

What is vllm?

Is vllm popular?

Is vllm well maintained?

Is vllm safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Fix identified vulnerabilities

Monitor for new issues

Package Health Score

Explore Similar Packages

Direct Usage Popularity

Security and license risk for latest version

Commit Frequency

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Readme

Easy, fast, and cheap LLM serving for everyone

About

Getting Started

Contributing

Sponsors

Citation

Contact Us

vllm dependencies

FAQs

What is vllm?

Is vllm popular?

Is vllm well maintained?

Is vllm safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Source Code

Open Source

Container

Config

Fix identified vulnerabilities

Monitor for new issues