A PACTful Agenda for Cloud Programming Research: (Invited Talk)
We have witnessed two decades of cloud computing research. Yet, programming the cloud remains a tedious task for both the application and cloud infrastructure developers: application developers need to consider various cloud deployment aspects as they write code, while infrastructure developers must determine how to execute and optimize code that mingles application semantics, fault tolerance, and hardware constraints, etc. all into a single program.
Yet, cloud programming is not an entirely new problem: initial commercial efforts at a programmable cloud have started to take wing recently in the form of “serverless” Functions-as-a-Service (FaaS). FaaS offerings allow developers to write sequential code and upload it to the cloud, where it is executed in an independent, replicated fashion at whatever scale of workload it attracts. But fundamentally, even “FaaS done right” is a low-level assembly language for the cloud, a simple infrastructure for launching sequential code, a UDF framework without a programming model to host it.
In this talk, I will describe our new Hydro project for next generation cloud programming research. The key idea behind Hydro is a new programming interface targeted for cloud applications, where we decouple cloud programming into four separate concerns, each with an independent language facet: Program semantics, Availability, Consistency and Targets for optimization (PACT). To use Hydro, programmers first specify the intended functionality of their application (i.e., its semantics) using a declarative language. Then, they describe the failure domains (i.e., availability) once their application is deployed to the cloud. Consistency is then specified, independent from the program semantics, by stating invariants that their application should satisfy (e.g., how concurrent writes to the same data should be handled). Finally, the target facet describes the platform that the application should be deployed, for instance the expected types of hardware, minimal network connectivity required, etc.
Implementing the Hydro stack will open up new research opportunities in query compilation, execution, and runtime monitoring, for instance using program synthesis and other learning-based techniques to drive PACT program compilation rather than a static pattern-matching approach targeted for a specific backend, a distributed execution framework for monotonic code, iterative refinement of programs, etc. Hydro is a joint project with Natacha Crooks and Joe Hellerstein. In this talk I will explain these ideas and describe the early progress we have made thus far, and the many exciting opportunities that lie ahead.