Pathway
A Python ETL framework designed for efficient stream processing, real-time analytics, large language model (LLM) pipelines, and retrieval-augmented generation (RAG). pathway enables building robust and scalable data workflows with support for complex, real-time data transformations and AI-driven analytics.
Pathway is built in Python, 63.5k GitHub stars, latest release v0.30.0.
When to use Pathway
Pathway is listed here as a Development project. The directory calls out Stream processing with low latency, Supports real-time analytics workflows, Integrated LLM and RAG pipeline support as capabilities associated with it.
Other recorded traits for Pathway include Flexible ETL pipeline design, Scalable for large data volumes.
Besides Development, this page also files Pathway under Analytics, Python, AI.
Pathway compared with
Records in this directory name Apache Beam, Apache Flink, Dagster, Prefect as products people compare with Pathway. That list is editorial metadata, not a claim that Pathway replaces each of them.
What the Pathway stats reflect
GitHub currently shows 63.5k GitHub stars, 1.6k forks, 38 open issues, latest tracked release v0.30.0. Star and activity counts here are a snapshot used as a proxy for community adoption, not a quality score.
Stats refreshed
- Language
- Python
- Latest Release
- v0.30.0
- License
- Other
Our Newsletter
Get new Development tools right in your inbox
Get short emails with useful development projects, releases, and repos worth watching.
Key features of Pathway
- Stream processing with low latency
- Supports real-time analytics workflows
- Integrated LLM and RAG pipeline support
- Flexible ETL pipeline design
- Scalable for large data volumes
Recorded alternatives to Pathway
Pathway resources
Pathway on GitHub
Frequently asked questions
What is Pathway?
A Python ETL framework designed for efficient stream processing, real-time analytics, large language model (LLM) pipelines, and retrieval-augmented generation (RAG). pathway enables building robust and scalable data workflows with support for complex, real-time data transformations and AI-driven analytics. This directory highlights Stream processing with low latency, Supports real-time analytics workflows, Integrated LLM and RAG pipeline support.
What language is Pathway written in, and what is the latest release?
Pathway is written primarily in Python. The latest release tracked on this page is v0.30.0.
How widely is Pathway used on GitHub?
Pathway has about 63.5k GitHub stars. It also has about 1.6k forks. Those counts are a snapshot of community attention, not a ranking of quality.
What do people compare Pathway with?
This directory records Apache Beam, Apache Flink, Dagster, Prefect as comparison points for Pathway.
Related tools
Prefect
Prefect is a powerful workflow orchestration framework for building, running, and monitoring resilient data pipelines in Python.
Deer-flow
DeerFlow is a community-driven Deep Research framework, combining language models with tools like web search, crawling, and Python execution, while contributing back to the open-source community. It enables enhanced research capabilities through integration with various technologies.
Langfuse
Langfuse is an open source LLM engineering platform providing observability, metrics, evaluations, prompt management, and more for LLMs. It integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more.
Llama
Inference code for Llama models, a versatile solution for running Llama model inferences efficiently.
Autogen
A comprehensive programming framework for building agentic AI applications, supporting various integrations and community interactions.
Awesome LLM Apps
A curated collection of awesome Large Language Model (LLM) applications featuring AI Agents and Retrieval-Augmented Generation (RAG) using OpenAI, Anthropic, Gemini, and open-source models.