YouTube Video mftCGzVtoqo  ## Metadata - Author: [[YouTube]] - Full Title: YouTube Video mftCGzVtoqo - Category: #articles - Document Tags: [[ai]] [[enterprise]] - Summary: YouTube Video mftCGzVtoqo - URL: https://www.youtube.com/watch?v=mftCGzVtoqo My name is Yas. Thank you for joining this session at Jupandara. Today I would talk I would like to talk share uh something completely new. This is a Japan born technology and this session is it first public reveal anywhere in the world. I will present a new paradigm called prompt oriented programming which we call POP. Now let me introduce uh myself. My name is Jan. I'm a chief architect of holding all AWS certifications um working at the Sonar office of a Japanese mobile operator. My expertise is cloud. I hold all as certifications, DevOps and agile software development. I've been involved in Jupa community for 20 years. This is today's agenda. So now let me introduce the AGS migration project which provides the background for pop that we are working on. The project is about migrating our internal approval system AGS to a new platform. But instead of just replacing the system, our goal is to introduce innovation with genative AI. Specifically, we aim to improve efficiency allowing flexible specification changes given this project background. Here are the key insights we gain. During early development in the first quarter, we faced multimodel inputs like attachments, images, and forms. We naturally [clears throat] shifted to letting LLM handle extraction and validation for the attachments. And we realized that rules don't have to be in if else statement in a traditional programming logic. They can be expressed directly in France described by natural language. So what does this insight look like in practice? Let's compare the traditional codecentric programming versus YAML first as a proposal. On the left is the traditional if else implementation. on the right. The same logic is declaratively expressed as a YAML. By the way, YAML is a simple human readable text format. As you all know here, the key difference is that one can be modified simply by editing YAML as an instance. No code changes or redeployment required. To make more complete, here is a real example of a P&L purchase and purchase and lease approval request in AGS. We upload some files such as a screenshot or PDF of the items to purchase. The LM extract key fields such as the item name, amount, price, tax, and shipping cost. A young defined validator checks the fields. The result is stored as a JSON followed by a human in the loop confirmation before the approval flow continues. That's the concept. Now, let me show you a demo by screencast. [snorts] For the first demo, let's make a purchase request. I'll type SL chat in the Slack random channel. The B present two buttons, purchase request and the business chip request. I click purchase request. Imagine I want to buy this MacBook Air. Here in the Amazon um purchase page, I drag and drop the screenshot directly into Slack. The bot automatically pares the content. On the right of the screen, it shows the bot console. While the bot is working, it is sending the screenshot directly to Open AI. The model runs OCR, extract the required fields and prepares the data for the form and it shows me a button show form. When I click show form, a Slack form is generated with all required field prefilled automatically. No manual entry. Everything is extracted from the screencast screenshot. That's it. For the second demo, let make a business trip request. I'm now actually speaking here about Pop at Jup here. Here is the conference session page and this is my session detail page. To request approval, I have prepared two documents. [clears throat] Screenshot of uh SFO to Kung Fu KIX right here. This is a file screenshot and a PDF from hotels.com for my hotel reservation including checking in and check out days and the cost. I drag and drop both into Slack and also I put Jupal N URL. Again, the bot automatically crushes them. On the right of the screen, it shows the B console too. While the B is working, it is sending the screenshots directly to Open AI. The model runs OCL, extracts the required fields and prepares the data for the form and it show the show form button. When I click it, a SAR form appears with all the detail already filled in. Hotel flight and its information, trip information. The request is ready to submit instantly and finally request clicks the submit button. So what you just saw was business logic is ex executed not through traditional programming code but through a combination of a YAML based framework and genative AI. That's why this is our brand new proposal prompts oriented programming or POP. Let me show you the difference. This diagram shows the contrast in between codeentric if else which is traditional web application and YAML first pop. In a traditional codecentric web application from integos through web UI then if else code then to HTML output with y first pop in to go through chat UI and then yamo blueprint through the area and self validation then to JSON for chat messages or HTML output. This illustrates how POB replaces static logic with blueprint driven processing. At my company, I explain this as traditional web application versus PP. But here at the Jupa. So let us move on to Jupa versus pop. This diagram shows Jupa and Yam first pop. In Jupal from form in codes through web forms and views which is rendered through Yam by Drupal. Then Jupyal core renders the YAML for the configuration. Then to HTML or JSON output with YAML first POP input to go through CH UI and then YAML groupings through the LM and self validation then to JSON for chat messages or HTML output. So again that is Jupal on the top and POP on the bottom. Both start from user input and both end with structures output. But what happens in the middle is similar in shape yet fundamentally different. Jupa processes everything through its web forms and views by YAML PHP code and Jupa core. Pop takes a Y blueprint instead feeds it a generative AI and let the runtime handle validation logic and rendering. The key point is that Jupal relies on static configuration and code while POP runs the logic directly from YAML. So when requirements change we update YAML not the code base that is the shift we that's the shift we call YAML first. So let's take a closer look at how POP and the PBF which stand for prompt based framework pipeline work. So pop means shifting from codeentric to Yamu first. Business logic is expressed as YAML blueprints which is equivalent a design document. MLM enables dynamic processing, validation and localization. The prompt based framework which we call PV pipeline handles everything such as blueprint LM self variation schema variation and output. And the system is extensible through problems [clears throat] following crystal clear design patterns behind the scenes. Let me briefly show the architecture. I will not go into every detail here but the key point is we have already thought through orchestration, validation and extensibility. It is not a concept but also a framework level design. So now what does it actually look like in yellow? Here is a yellow example as a blueprint. On the left we have a template that define a structure and on the right an actual example applied to our use case. The role section is especially important when we explicitly tell the LLM you are a validator or you are an extractor. The model adopts that [clears throat] role and produces outputs more consistently. In fact, while developing our framework, we realized that much of business logic can be expressed through a set of basic laws such as extractor, classifier, [clears throat] summarizer, validator, verifier, converter, translator and writer orchestrators. These roles serves as reusable building blocks in prompt oriented programming. In the scenario section, the instructions are written in a sty gin syntax using given, when. Although is normally used in BDD testing, here we use the same syntax for prompt design. Large language models understand this pattern because they were pre-trained on material that includes girking style text. That pre-rain is literally the key in chip. This is why the structure works so well and keeps the model's reasoning clear and consistent. Output output schema and prompt example also guide the LM toward consistent results. The point is what used to be hardcoded logic is now declaratively written in Java. Now let me show you how we generate a pop blueprint from existing documents. We build a YAML blueprint builder that reads attachments like the existing Microsoft Office files and or design documents send them to the AI automator and produces clean YAML blueprint. Here is a demo. I start signing in just like a normal Juba site. From the standard content menu, I create a new YAML icon. I give it title and upload a reference file. This file contains routing rules for approval workflow. Once the file is uploaded, I save it and Juba extracts the text from the uploaded document. Let the system process the content. It is now extracting the routing patterns turn them into structure summary and then produces POP blueprint. The blueprint becomes executable as it is. No manual coding is needed since the logic is expressed directly in yellow. This is a pop model in action inside Drupal. Here is the output as the yellow blueprint. The LM generate the prompt from the attachment and define the output schema based on JSON format. So as we could see the demo, Juper has always been about configuration defining how systems behave through YAML and programs. Pop takes that same idea and extends it to cognition defining how AI should think. In other words, Jupal build a foundation. Pop just bring it into the AI era. Of course, there are still challenges for the issues and the risks risks to address technically LM hall and runtime overhead are concerns from a from a governance perspective. Audit and compliance frameworks are still underdeveloped and security requirements such as ARA and PI masking needed need reinforcement. Strategically we must differentiate POP from similar frameworks like a launch chain or Azure front flow. Our approach is to emphasize pop as reusable design patterns for enterprise system. [clears throat] Despite these risks, our PC already shows clear benefit from our P proof of concept. We saw four main benefits. Workflow changes are reflected instantly by editing YAML. Another important benefit is that user interface can be autogenerated directly from YA. For example, from our output schema in YAML, we can automatically build a snap form as we could see the demonstration. This makes it easy to keep the UI and the workflow logic in sync without manual coding. Safe evalariation with human in the loop reduce output errors. Finally, we compared with legacy system. We reduce the overall demo op effort in the old environment. Every rule change required code edit, CI/CD and redeployment. With POP, most of work is replaced by simple YAML edits and version control. This significantly lowers the operational overhead. Looking ahead, so where are we heading next? Here's our future direction. POP will be fully adopted in the AGS migration internally in my company and we will define approval workflows directory in YAML extend functions as plugins and adopt run for dynamic grouping selection in the future. This is not just a technical choice but also a shift new paradigm of software development with genative AI at the foundation. So with this road map in mind let me conclude with our vision. AGS migration is not just a system refresh but also the first step of pop adoption. The benefits of pop are first faster feedback to specification changes. Rules can be updated by simply editing yellow. Second, flexible handling of multiple diverse inputs and outputs which allows the system to adapt across different use cases. And the third expected efficiency gains in both development and operations compared with traditional codecentric systems. Ultimately, we aim to establish POP as reusable design patterns as a standard programming framework for future enterprise system and application devel development turning migration challenge into an innovation opportunity. Lastly, I have a question for you. Is this an innovation? That concludes my presentation. Thank you for your attention. Any questions? Yes. >> Uh looks very interesting. have a question about uh how do you uh receive feedback from LLMs when uh someone uploads the bad uh screenshot of missing information. So for example, >> we have all the fields except the uh amount of items and should give you together with the text transfer it should give you a feedback. Okay, this set 50% of the fields okay but we have problem with this field that field that field and we should really ask to upload image or field manually. How can we handle this? >> Yeah, actually well the basic idea is we scatter the fields uh with the YAML uh prompts so the can you know easily process the field by field. So uh I bet that so um according to our uh P almost 100% sure to get the accur um accurate information from the error because we scattered the small logics not like you know recently everybody is saying the VIP coding so VIP coding is you know generates [clears throat] the one huge application Right. So sometimes has some hination kind of thing. Right. But this approach is um we scatter the logic with the some small chunks of the prompts and then we send the prompts to the other. >> Yeah. I would uh I tried the same approach but it's also a little n even if I ask uh lab to produce the result in uh y in JSON file it's sometimes forgot the comma sometimes forot something else open in brackets close brackets and we got the a broken JSON in the middle and everything broken >> oh okay so that's why uh we have some out schema maybe you have the same approach But um in order to make sure the more higher accuracy we define the schema with some this kind of the JSON format kind of thing. >> Uh so this is the input prompt for >> no this this is input prompt. >> Yeah with input uh inside input prompt we put the output schema format. >> Okay. [clears throat] So M will follow this format >> and response of the LM is YAML formatted or text formatted. So when you have this prompt as the output you receive always JSON or in what format to make it possible by code or by something else. >> Oh yes. So we can we can change the JSON easily. Uh we can change the JSON or some other CSV. >> Yeah. Yeah, the output schema is exactly what you expect from the LM, right? The ces of JSON with this format. >> Yes. >> And it works. Yeah. >> Yeah. So, Ed can understand this one. >> For me, it's maybe 10 to 20% times broken JSON, missing fields and so on. And I can find helpful fixes. Well, then so uh uh according to our tri well I'm sure that you know everything can deter the um >> exactly the >> exact [clears throat] cable play. >> Yeah. So we try to make the accuracy more higher that's why we take [clears throat] some the format the young format like as I said like starting from the g [clears throat] to the and also we put some scenario structure like a given when and also we have a out schema section and then example so uh you can follow the example so We are trying yeah we understand that is always probability so we cannot expect the 100% sure accuracy right so we are trying how we can get the uh the more 99% accuracy from the area yeah >> yeah thank you >> thank you >> any other question yes >> yeah thanks for the really interesting [clears throat] presentation um and I can See, I really see the value of having an LLM process the input files and prepare [clears throat] a form that the the user can then correct in case some of the values are are wrong or something got misunderstood. >> I really see the value of that. >> And then I I also understand the value of having your LLM process your business rules from the document. >> Yes. But I wonder if it might be more less costly and more reliable to have then your model turn those business rules into either ECA models or custom code >> that makes sure that those things get processed the same way every time. >> Yes. Yes. Yes. Yes. That's the another idea. So uh once we can give the LM LM can create some small snippet it's more static code like a traditional programming model maybe that is a hybrid approach but uh uh my message is the B again by coding is too large so we won't we will try to split the split our problem into the more smaller you know program and then L can easily you know handle even the even the previous generation model L model can handle this kind of thing. >> Yeah. Thanks. Thank you. So have you have you um tested the system against um all the known um prompt injection >> or prompt injection? >> Yeah. Yeah. So I was a bit um yeah I thought you know I mean for example your validators may be able to pick up some of those but >> [clears throat] >> um sometimes some sophisticated um prompt injection can um how do you say get get through that validation and maybe you know you can you can have some kind of script not script prompt in like a a screenshot or something for example and I was wondering if you've tested Oh, >> we we haven't tested uh yet. So, so you Yeah, that that's a good point. I guess you know uh you're talking that um if they uh some user put some attachment or screenshot with your some kind of prompt injection kind of information. >> Uh we we haven't tried that >> photo database. I see. Yeah, that's maybe yeah, we need to >> Yeah, maybe just Yeah, for for your next >> I see. I see. Yes. Yes. Yes. Thank you so much. Yeah, you are insight. >> I think it should not be a problem because if you read text in the that is not execute just will in result you will got some not accurate results. >> Yeah, I guess it depends on how the system is structured really. if it requires like 50 scores in in >> yeah that that's a good point that basically uh we expect the output schema so looks like out of schema will define some kind of a string or integer it's more like a strict you know the traditional type of programming model but so um every will fill out those values from the attachment, right? So, promptly from the point of view of the uh prompt injection, yeah, we need to consider about that from the security concerns. Yes, thank you so much for your insight. [clears throat] >> Yes, if you have time question about receiving the several results from one problem. For example, if you upload a screenshot, uh the LLM can fill the form >> and also I need to get the information about how many fields are missing. uh is uh it's better to make a separate call to analyze the so as a one result we should make the uh field form should receive and other part we should receive an integer number of field to field correctly uh is it better to make one uh or separate one to fill the form and another is to calculate what is missing >> I see so yeah you you your question is how how large we can give the LM. >> Yeah. So is it better to make one request with complex tests together or separate request because separate request will take more time, more money. >> Yes. Yes. Yes. Uh it is depending on the the the uh we have some [clears throat] uh several um top topics to talk about. So one need how large the area can handle at the context window right. The other thing is uh the cost right the latest model the cost lots of you know the lots of charge. So um I mean but [clears throat] according to our trial the variation of the L model we can use the more smaller L model because we will uh split the uh our prompt into the small chunks right so [clears throat] but still um so answer to your question um in our enterprise you know approval system it it's okay you know it's acceptable but I'm not sure that it's depending on the use case and the forms yeah >> yeah maybe an easier use case you put an article and run two tests the first test is to create a short title of the article and the second test is categorize is it about nature is it about techniques is it about programming so there are two tests and it's is it okay to ask them to put to perform both tests in one uh uh prompt or separate prompt. [clears throat] >> Yeah. So we can scatter the YAML blueprints to give the uh uh error. Yeah. >> And this will be two separate pros to >> yeah two separate in into the one file and then runtime can handle to scatter uh those ones and >> so we can put >> put it together >> but yeah we are developing a runtime for the pop. So runtime like a Java VM right can run the uh uh intermediate code like class file. So that's the same idea actually. So instead of using the Java code we uh we define the structure of the the Y and then runtime will take those sections. >> So those two scenarios will be executed as one prompt if if it's possible. Yeah, maybe. Well, in the future if the user give us the long Yahoo file, the one time automatically humble to separate Yeah. separate those scenarios [clears throat] one by one. >> It's not just control. >> Yeah. So, it's depending on the how smart and round. So we are trying to develop you know that's part of the runtime you know that needs to have also cash or you know as R said that you know we can uh give that capability to generate the small snippet called gen generator into the runtime and cach it or you know um if the runtime is smarter and smarter then uh the yu file we can [clears throat] accept the any young file even young file is longer >> so that's a future [clears throat] and also and I I don't worry about the the response time or performance you know after 10 year or 20 year later and then you know easily uh respond those outputs right yeah okay [clears throat] >> any other questions I do one. >> Okay. >> I was just curious when you are say you have a process that you've already defined with your YAML generator >> and then you want to refine that process. >> Do you go through and upload an updated version of that file? Do you go through and edit the YAML that got generated directly? How do you handle like maybe I saw one I saw one thing in there if the amount is over $30,000 treat it differently or something like that >> if you if the business logic changes >> and you need to do something different how how do you do >> how do you make that change? Do you do it directly in the YAML or do you >> uh directly in the YAML a description like uh [clears throat] here the web is an action basically. So uh we can put some rules. >> Okay. >> Yeah. [clears throat] >> So once you generate it from the LLM from then you you start manually. >> Yes. Okay. Yes. But in the future, I um envisioning that um uh currently we build the YAML blueprint builder manually but um based on this framework uh we can shift this BOP to a kind of a bipolic structure. You you understand you know kind [clears throat] by coding is to generate the entire application but given giving the this kind of the framework and specification to the area can understand this pop concept everything and then uh we can get the more accurate more uh high quality uh result from the area. Yeah, [clears throat] that's my vision. Yeah, any other questions? Okay, so nice discussion. Thank you so much for attending., YouTube Video mftCGzVtoqo, YouTube, 01kabxsmxmnajcsasxtbmyyr1h.