Jupyter Deploy: The New Middle Ground Between Laptops and Enterprise  ## Metadata - Author: [[The New Stack Podcast]] - Full Title: Jupyter Deploy: The New Middle Ground Between Laptops and Enterprise - Category: #podcasts - URL: https://share.snipd.com/episode/30ff83c6-4b5d-4e8c-b653-81839924a285 ## Document Notes ## Highlights - **Solving The Middle Ground Problem** - Jupyter Deploy targets users stuck between local laptops and enterprise-managed services who need cloud compute and collaboration. - It fills a middle ground by automating complex cloud scaffolding so small teams can run Jupyter securely in the cloud. Transcript: Jonathan Guinegagne So we've heard from a lot of users that they want to run Jupyter on the cloud and for various reasons. Some of those reasons are they just need more compute. Some of the other reasons is they want to collaborate with distributed teams. And in order to do that, running it on your laptop is not going to do it because you cannot just open up your laptop ports and provide access to other users over the Internet to your own laptop. So we heard this multiple times. And what we found was that there was a lot of users can run it either on their laptop or they are managed services by the big cloud providers that provided a fully managed experience for Enterprises. And there's really not much in between. ([Time 0:01:26](https://share.snipd.com/snip/0cd70d3d-f20e-4c96-b6d5-445b39bddac4)) - **Use And Extend Templates** - Use templates to make Jupyter Deploy extensible across clouds, identity systems, and compute choices. - Contribute templates to adapt deployments to other providers, on-prem, or Kubernetes clusters. Transcript: Heather Joslin What is the target audience for Jupyter Deploy? Jonathan Guinegagne So we are starting with small teams that need to collaborate in order, for example, to do a demo with a few collaborators across the world, or a small team of researchers, educators, Fairly small team, less than 10 individuals would want to cooperate. And what we hear and what I heard, for example, during this conference is that people will just go through the whole process themselves. They will set up something, they will spin up a server, they will run something on some form of compute instance. So the idea is, how can we make this easier? How can we make this secure by default? We're bringing AWS expertise in security and generally our knowledge of our own cloud infrastructure. And then we recognize that AWS is one of the cloud providers. So if we can make this tool open source of the community can contribute over what we call templates so over deployment recipes to make it work with our cloud provider perhaps or other Form of authentication that yeah we will welcome those kind of contributions. ([Time 0:04:30](https://share.snipd.com/snip/cc76b277-ce32-43ef-b61f-075613c1e62d)) - **Education And Small Teams Demand Simplicity** - Educators and workshop hosts asked for easy cloud Jupyter to provide GPUs and avoid signup hoops for students. - Small research teams and startups also requested simpler multiuser setups than JupyterHub to reduce operational burden. Transcript: Jonathan Guinegagne We've heard during this conference, educators who just want to show it in a classroom. People who run workshops, they want to teach data science or to teach machine learning. And Jupyter is a wonderful tool for this. It's interactive. It can run a lot of compute. It can display rich media, et cetera. So they like it. They have the problem of how can we get the GPUs that we need, for example, when you want to run deep learning workflows or deep learning examples. And how do you make it accessible without having too much setup and without having people go through many hoops to sign up etc. And so that would be one of the use cases the other use case we heard is, is there you are, you are startup or you are an organization that requires their the multiple user to come in and spin Up their own compute instance on a generic framework. So that's typically what JupyterHub has done in the past. But what we've heard from some customers is that it's not easy to set up JupyterHub, and a lot of the customers and a lot of the users need help there. So is there a way to simplify this? ([Time 0:06:54](https://share.snipd.com/snip/6634067d-e346-4ac4-a4a3-d6b26f472cba)) - **Path Toward Jupyter Governance** - Project governance aims to join the Jupyter ecosystem with leadership from Brian Granger and eventual governance by the Jupyter project. - The team recently opened the repo and expects the CLI to integrate into the official Jupyter tooling over time. Transcript: Jonathan Guinegagne So, yeah, so that's an interesting question. The leader of our team is Brian Granger, who was one of the original founders of Jupyter. So his vision for it is to be accepted by Jupyter console at some point and become governed by the Jupyter console. For now, we are developing it. We recently made it publicly available on GitHub. That's very recent, like only a couple of weeks back. So for now, it's mostly our team having access to it. But this is the vision that it becomes part of the Jupyter ecosystem. ([Time 0:10:23](https://share.snipd.com/snip/587f22fc-9f25-4063-b8f2-0d14b759d736)) - **Terraform Enables Portability** - Jupyter Deploy uses Terraform as its base to remain vendor-neutral and support multiple clouds and Kubernetes. - Terraform providers let templates target AWS, Azure, native Kubernetes, or other platforms without rewriting the CLI. Transcript: Jonathan Guinegagne Yeah, so when we looked at Jupyter Deploy, what we saw, even for fairly simple templates with a single EC2 instance that routes connection over to a domain and using GitHub authentication, We needed something like 15 or 20 cloud resources. Just because there's a lot to it. And so when we started looking at this, we started, OK, we need infrastructure as code pretty quickly in order to manage this. It's not something where you can make a couple CLI commands to set up your infrastructure. It's not going to be enough. And then when we look at, from our experience working with our customers at AWS, has solutions for infrastructure as code, but our customers really like Terraform or infrastructure As code solutions that are vendor neutral. So based on that experience that we have and what we've heard from our users and our customers, that's what we decided to use. So our base templates use this Terraform, which is vendor-neutral. So Terraform has a provider setup for Azure, for like all the cloud provider for AWS, and even for native Kubernetes, it can make, it can read your kubeconfig and deploy to native Kubernetes. ([Time 0:11:20](https://share.snipd.com/snip/6081ef3c-cfe0-4614-a005-4448d4702051)) - **Plan For Growing Cloud Expertise** - Treat Jupyter Deploy as an on-ramp, not a long-term magic solution for complex production needs. - Learn cloud fundamentals as you scale, because running GPUs and multiuser systems will require deeper expertise. Transcript: Jonathan Guinegagne So it gets you started but at a certain point running things on the cloud will require you to understand what are the components behind it. So it gets you from not knowing anything about the cloud and how to even begin to a deployment that's working. Then if you need to evolve it and if you need to manage an infrastructure that has several users, that uses GPU, things are going to start crashing, things are going to start to go bad. This is where you will need more expertise and you will need to learn those things. So this is useful for people to get started, people to not start from nothing. It's not a magic wand that will solve all your cloud problems. ([Time 0:13:24](https://share.snipd.com/snip/ced87666-df14-497f-8bd4-fcb3be9184ba)) - **Lightweight Containerized Stack** - The deployed stack uses Docker, Docker Compose, and fast Python dependency tools (pdm and micropipenv alternatives) for reproducible images. - It integrates Pixie for Conda-compatible scientific packages and builds images quickly for education and research use. Transcript: Jonathan Guinegagne Use Docker in order to run our services. So the base template essentially sets up an EC2 instance. So in Kubernetes, Lingo, that would be a node. So a host. And on this host, you run services as Docker services in Docker Compose. So that we use Docker. And the image that we use are very simple images that are built on the fly as well. And we use UV for the dependency management. And if you haven't tried UV, I really recommend it. It's much, much faster in terms of resolving the dependencies. As well as, and this is the feedback we got with working with educational organizations and universities, UE is good, but it doesn't integrate with Conda. And for some of the scientific package, you need Conda. So there's this other dependency management, which is Pixie, which we use and integrate as well in the base template so that you can essentially, your image is based on Pixie and you Can do Pixie add and add some dependencies that come from conda forge or from other channels as well as from UV and building them very fast. Infrastructure we use. ([Time 0:15:20](https://share.snipd.com/snip/a56e2d35-8e61-4ff0-8f78-1414a7eefb92)) - **Enable Auth And TLS By Default** - Secure deployments by default: use OAuth2 proxy, Traefik, and Let's Encrypt for authentication and TLS termination. - Deploy these middleware services via Docker to ensure end-to-end encryption and managed ingress. Transcript: Jonathan Guinegagne And then in order to provide integration with OAuth and with GitHub identity, we use OAuth2 proxy, which is middleware that you can deploy. And so we deploy it as a Docker service, which will talk to GitHub, essentially, in this case, to provide the authentication and verify whether that particular GitHub identity is authorized To access your app. Lastly, we use Traffic for proxy. So Traffic has a free open source version, has a premium service for enterprises. So we use the free open source version there as well. As a way to do the entry point, generate the, do the TLS termination, handle the ingress. Finally we use let's encrypt to vend uh tls certificate to generate the tls certificate and provide them this is a really amazing tool it's free it works very well uh to to generate a certificate And that gets you end-to encryption all the way till to your um without relying on anything. ([Time 0:16:47](https://share.snipd.com/snip/f3812165-612c-4097-ab08-2212ed5e1b8f))