March 5, 2025

AI4EOSC introduces AI4OS-LLM, allowing users to deploy their own Large Language Models to support research and development

The AI4EOSC project launches the AI4OS-LLM service, a new platform tool to deploy dedicated large language models (LLM), which is now available on the AI4EOSC Marketplace. This AI service is designed to support researchers and developers by providing advanced language processing capabilities by means of self-deployed LLM models, accessible via an API or a chatbot service. These models can be deployed and used in a transparent and accessible way.

This new functionality allows users to access advanced text generation, summarization, translation and conversational AI options without sharing private data. For this purpose, users can choose from several state-of-the-art small and medium-size models optimized to run on our platform for generative AI, such as DeepSeek-R1-Distill-Qwen-1.5B, Qwen2.5-7B-Instruct-AWQ or Llama-3.2-3B, among others. 

Following the cloud-based approach which guides the development of the AI4EOSC platform, this functionality does not require any installation step from the user side, making the deployment of generative AI models easier for users. For greater flexibility, this new tool offers the ability to deploy a chat bot user interface (based in Open WebUI), an API  backend endpoint (based in vLLM) or a combination of both services.

Why this service?

One of the main advantages of deploying your own LLM service through the AI4EOSC platform concerns privacy and transparency.

When deploying your own LLM instance you can retain the control over your data and queries, also reducing dependence on proprietary AI solutions. Note that no data will be collected by the platform, as all your information can easily be removed by deleting the deployment.

Try to deploy your own LLM from the AI4EOSC dashboard: https://dashboard.cloud.ai4eosc.eu/marketplace/tools/ai4os-llm