Leveraging open-source LLMs for production
This talk examines using open-source LLMs for real-world purposes. It compares the benefits and drawbacks of open-source LLMs to proprietary options like OpenAI. The discussion covers the economics of hosting open-source LLMs, highlights serving frameworks, explores cloud GPU availability, gives an overview of key open-source LLMs, and shows how to
This talk examines using open-source LLMs for real-world purposes. It compares the benefits and drawbacks of open-source LLMs to proprietary options like OpenAI. The discussion covers the economics of hosting open-source LLMs, highlights serving frameworks, explores cloud GPU availability, gives an overview of key open-source LLMs, and shows how to run and refine these models using the dstack open-source tool.
Speaker:
Andrey is the founder of dstack, an open-source toolkit for orchestrating GPU workloads across clouds. Prior to this, Andrey worked at JetBrains with the PyCharm team.
