Tech journey promo

There are some tech journey events coming up soon that I would like to share here. It is a series of Webinars, in german, all remote. If you are interested, you should register immediately.

  • 04.06. - Storage integration in OpenShift Virtualization
  • 11.06. - Erste Schritte mit Gen AI auf Red Hat OpenShift
  • 27.08. - OpenShift Virtualization 201: Day 2 Operations in der Praxis

On top of that, there’ll be a dedicated AI Roadshow, in Person. Here are the dates (subject to change):

  • 03.07. → in Frankfurt, by Nicolas Wieske
  • 10.07. → in Munich, by Thomas Herath

If I get more information, I will let you know.

AnythingLLM and the RAG use case

AnythingLLM is an open-source application designed to make working with large language models easier and more productive, especially when dealing with custom or private data. It acts as a local, customizable interface that allows users to chat with an LLM using their own documents, data, or knowledge base — without needing to send sensitive information to external servers. In essence, it is an AI-powered assistant where you can do Retrieval-Augmented Generation (RAG) in a very simple way.

OpenShift AI (RHOAI) is Red Hat’s platform for developing, deploying, and managing AI/ML models at scale. It provides End-to-End ML Lifecycle Support: From data preparation and model training to deployment and monitoring, includes Jupyter notebooks, model serving tools, and popular ML libraries. In essence, it is the management platform for workbenches, models, data storage, monitoring and ML pipelines.

Model as a Service (MaaS) is a cloud-based approach that provides machine learning models as on-demand services. Instead of developing, training, and hosting models from scratch, users can access pre-trained or custom models via APIs. It speeds up integration of AI into applications and reduces the need for in-house ML infrastructure. In short, MaaS simplifies the use of machine learning by abstracting away the complexities of model management and deployment.

Why am I writing this? Good question.

As already described in CW14, I am currently working on several proof of concepts around OpenShift AI and one of the many use cases being investigated is that of a RAG chatbot.

So the question is, what is the best way to show that everything is stable and works well? AnythingLLM is very suitable for this. You can deploy it very easily as an OpenShift AI workbench. AnythingLLM also comes with its own integrated vector database and an embedding model, but you can also connect your own instances. In OpenShift AI, you can also easily deploy models and make them available using internal or external endpoints.

To map the use case, you can then connect your own LLM in AnythingLLM and upload your own documents. These are vectorized and used as a vector when a request is made to the LLM, which fulfills the RAG use case.

The easiest way to check it out by yourself is using things included in AnythingLLM and use a MaaS as LLM. At Red Hat we have an own super simple MaaS which is used in the Parasol Demos, demos showing all kinds of things on OpenShift AI. You can try it yourself and get your credentials here.

For more information, here are some additional links: