I'm also curious about this. The folks I hear about Temporal from seem to be very disjoint from Airflow users, and Temporal's python client is still alpha-stage.

It seems notable to me that the big Prefect rewrite mentioned elsewhere [0] leans into the same "workflow" terminology that Temporal uses. I have to wonder if Prefect saw Temporal as superceding the DAG tools in coming years and this is them trying to head that off.

That post's discussion of DAG vs workflow also sounds a lot like why PyTorch was created and has seen so much success. Tensorflow was static graphs, pytorch gave us dynamism.

[0] https://www.prefect.io/blog/announcing-prefect-orion/

I know airbyte.io (elt platform) is built on top of Temporal, but I haven't used it.

Yes, Airbyte is using Temporal. Here is a blog post they wrote a few weeks ago that goes into more detail about it: https://airbyte.com/blog/scale-workflow-orchestration-with-t...

I’d love to hear that too :)

Hi, Tom from Temporal here. I don't have a lot of experience with Apache Airflow personally, but I was at Cloudera when it was added to our Data Engineering service, so I learned about it at the time. Here are a few things that come to mind:

* Both Apache Airflow and Temporal are open source

* Both create workflows from code, but the approach is different. With Airflow, you write some code and then generate a DAG that Airflow can execute. With Temporal, your code is your workflow, which means you can use your standard tools for testing, debugging, and managing your code.

* With Airflow, you must write Python code. Temporal has SDKs for several languages, including Go, Java, TypeScript, and PHP. The Python SDK is already in beta and there's work underway for a .NET SDK.

* Airflow is pretty focused on the data pipeline use case, while Temporal is a more general solution for making code run reliably in an unreliable world. You can certainly run data pipeline workloads on Temporal, but those are a small fraction of what developers are doing with Temporal (more here: https://temporal.io/use-cases).

Do you see Temporal as being a super-set of DAG managers like Airflow/Dagster/Prefect, or do you see uses where those tools would be a better choice than Temporal?

All three are relatively similar, in terms of being designed around a DAG model and focused on data pipelines. If your workflow fits neatly into that box and your team already has experience using one of them successfully, then the effort involved in switching might outweigh the gain.

While my experience with those specific tools is limited, I have used other DAG-based systems (notably Apache Oozie) quite a bit. I can't think of anything I did with those that I couldn't do with Temporal.

Temporal is distinct in that it's neither limited to DAGs nor to data pipelines. I suppose it is therefore a super-set, even though I don't really think of it that way (much as I don't think of Go or Rust as a super-set of bash).

while you're here, we'd love for Temporal to have a way to control max concurrent executions of a given workflow type!