<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
    <title>vigoo&#x27;s software development blog - mcp</title>
    <link rel="self" type="application/atom+xml" href="https://blog.vigoo.dev/tags/mcp/atom.xml"/>
    <link rel="alternate" type="text/html" href="https://blog.vigoo.dev"/>
    <generator uri="https://www.getzola.org/">Zola</generator>
    <updated>2026-04-11T00:00:00+00:00</updated>
    <id>https://blog.vigoo.dev/tags/mcp/atom.xml</id>
    <entry xml:lang="en">
        <title>Golem 1.5 features - Part 3: MCP</title>
        <published>2026-04-11T00:00:00+00:00</published>
        <updated>2026-04-11T00:00:00+00:00</updated>
        
        <author>
          <name>
            
              Daniel Vigovszky
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://blog.vigoo.dev/posts/golem15-part3-mcp/"/>
        <id>https://blog.vigoo.dev/posts/golem15-part3-mcp/</id>
        
        <content type="html" xml:base="https://blog.vigoo.dev/posts/golem15-part3-mcp/">&lt;h2 id=&quot;introduction&quot;&gt;Introduction&lt;&#x2F;h2&gt;
&lt;p&gt;I am writing a series of &lt;em&gt;short&lt;&#x2F;em&gt; posts showcasing the new features of &lt;strong&gt;Golem 1.5&lt;&#x2F;strong&gt;, to be released at the end of April, 2026. The episodes of this series will be short and assume the reader knows what Golem is. Check my &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;blog.vigoo.dev&#x2F;tags&#x2F;golem&#x2F;&quot;&gt;other Golem-related posts&lt;&#x2F;a&gt; for more information!&lt;&#x2F;p&gt;
&lt;p&gt;Parts released so far:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;&#x2F;posts&#x2F;golem15-part1-code-first-routes&quot;&gt;Part 1: Code-first routes&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a href=&quot;&#x2F;posts&#x2F;golem15-part2-webhooks&quot;&gt;Part 2: Webhooks&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a href=&quot;&#x2F;posts&#x2F;golem15-part3-mcp&quot;&gt;Part 3: MCP&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a href=&quot;&#x2F;posts&#x2F;golem15-part4-nodejs&quot;&gt;Part 4: Node.js compatibility&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a href=&quot;&#x2F;posts&#x2F;golem15-part5-scala&quot;&gt;Part 5: Scala support&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h2 id=&quot;mcp&quot;&gt;MCP&lt;&#x2F;h2&gt;
&lt;p&gt;&lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;modelcontextprotocol.io&#x2F;docs&#x2F;getting-started&#x2F;intro&quot;&gt;MCP (Model Context Protocol)&lt;&#x2F;a&gt; became a standard way to connect AI applications. With the new Golem release any Golem application can be automatically &lt;strong&gt;exposed through MCP&lt;&#x2F;strong&gt;. It does not require any code written, MCP is available for any agent automatically, but it needs to be enabled in the &lt;strong&gt;application manifest&lt;&#x2F;strong&gt;:&lt;&#x2F;p&gt;
&lt;h3 id=&quot;enabling&quot;&gt;Enabling&lt;&#x2F;h3&gt;
&lt;p&gt;The same way how we can deploy HTTP APIs with the application manifest, we can add an &lt;code&gt;mcp&lt;&#x2F;code&gt; section and choose which agents to deploy to which subdomains per environment:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;mcp&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  deployments&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    local&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; domain&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; mcp-demo.localhost:9007&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;        agents&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;          CounterAgent&lt;&#x2F;span&gt;&lt;span&gt;: {}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h3 id=&quot;security&quot;&gt;Security&lt;&#x2F;h3&gt;
&lt;p&gt;The above manifest section already exposes the listed agent through MCP, but it is not protected by any form of authentication. Similar to how we can protect HTTP endpoints with OAuth, we can attach a &lt;strong&gt;security scheme&lt;&#x2F;strong&gt; to our MCP deployment, let&#x27;s call it &lt;code&gt;mcp-oauth&lt;&#x2F;code&gt;:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;mcp&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  deployments&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    local&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;      -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; domain&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; mcp-demo.localhost:9007&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;        agents&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;          CounterAgent&lt;&#x2F;span&gt;&lt;span&gt;: &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;            securityScheme&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; mcp-oauth&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;To set that up we need an OAuth provider and have to create a security scheme using the &lt;code&gt;golem&lt;&#x2F;code&gt; CLI. The provider can be one of the common ones like Google, etc., or any custom one. In this post we are going to use the &lt;code&gt;mock-oauth2-server&lt;&#x2F;code&gt; docker container:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;docker&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; run&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt; -d&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --name&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;golem-mock-oauth2&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  -p&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;9099:8080&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;  ghcr.io&#x2F;navikt&#x2F;mock-oauth2-server:2.1.10&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;CLIENT_ID&lt;&#x2F;span&gt;&lt;span&gt;=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;golem-mcp-client&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;CLIENT_SECRET&lt;&#x2F;span&gt;&lt;span&gt;=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;golem-mcp-secret&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;REDIRECT_URL&lt;&#x2F;span&gt;&lt;span&gt;=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;http:&#x2F;&#x2F;mcp-demo.localhost:9007&#x2F;mcp&#x2F;oauth&#x2F;callback&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;golem&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt; -L&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; api security-scheme create&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --provider-type&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; custom&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --custom-provider-name&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;mock-oauth2&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --custom-issuer-url&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;http:&#x2F;&#x2F;localhost:9099&#x2F;golem&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --client-id&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;${CLIENT_ID}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --client-secret&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;${CLIENT_SECRET}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --scope&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; openid&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --scope&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; email&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --scope&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; profile&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;  --redirect-url&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;${REDIRECT_URL}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;  mcp-oauth&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;With that set, and running &lt;code&gt;golem deploy&lt;&#x2F;code&gt;, our MCP server is ready to be used at &lt;code&gt;http:&#x2F;&#x2F;mcp-demo.localhost:9007&#x2F;mcp&lt;&#x2F;code&gt; using &lt;em&gt;streamable HTTP&lt;&#x2F;em&gt; protocol, authenticated by the mock OAuth2 server.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;demo&quot;&gt;Demo&lt;&#x2F;h3&gt;
&lt;p&gt;We can prove this by simply creating the default template (which simply implements a stateful counter), let&#x27;s use the Rust one for this example and modify it slightly:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[agent_definition(mount &lt;&#x2F;span&gt;&lt;span&gt;=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;&#x2F;counters&#x2F;{name}&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub trait&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; CounterAgent&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt;    &#x2F;&#x2F; The agent constructor, its parameters identify the agent&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;    fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; new&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;name&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; String&lt;&#x2F;span&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; Self&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;    #[description(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increment by a given number&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;    fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; increment_by&lt;&#x2F;span&gt;&lt;span&gt;(&amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; n&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;struct&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; CounterImpl&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    _name&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; String&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    count&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[agent_implementation]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;impl&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; CounterAgent&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; for&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; CounterImpl&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;    fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; new&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;name&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; String&lt;&#x2F;span&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; Self&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;        Self&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;            _name&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; name&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;            count&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt; 0&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;    fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; increment_by&lt;&#x2F;span&gt;&lt;span&gt;(&amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; n&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;        self&lt;&#x2F;span&gt;&lt;span&gt;.count +=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; n&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;        self&lt;&#x2F;span&gt;&lt;span&gt;.count&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;We replaced the default &lt;code&gt;increment&lt;&#x2F;code&gt; method with a parametrized &lt;code&gt;increment_by&lt;&#x2F;code&gt;, which is going to be mapped into an &lt;strong&gt;MCP tool&lt;&#x2F;strong&gt; in our MCP server automatically.&lt;&#x2F;p&gt;
&lt;p&gt;If we start the &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;modelcontextprotocol.io&#x2F;docs&#x2F;tools&#x2F;inspector&quot;&gt;MCP Inspector&lt;&#x2F;a&gt;:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;npx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; @modelcontextprotocol&#x2F;inspector node build&#x2F;index.js&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;&lt;img src=&quot;&#x2F;images&#x2F;golem15-mcp1.png&quot; alt=&quot;&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;then click on the &lt;em&gt;quick auth flow&lt;&#x2F;em&gt; and &lt;em&gt;connect&lt;&#x2F;em&gt;, we can go to the &lt;em&gt;Tools&lt;&#x2F;em&gt; page and see our counter incrementation tool:&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;&#x2F;images&#x2F;golem15-mcp2.png&quot; alt=&quot;&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;We can pass the &lt;code&gt;Counter&lt;&#x2F;code&gt; agent&#x27;s constructor parameter, &lt;code&gt;name&lt;&#x2F;code&gt; and the &lt;code&gt;increment_by&lt;&#x2F;code&gt; method&#x27;s &lt;code&gt;n&lt;&#x2F;code&gt; parameter, and invoke it through an MCP tool.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;mapping&quot;&gt;Mapping&lt;&#x2F;h3&gt;
&lt;p&gt;As demonstrated above, &lt;code&gt;increment_by&lt;&#x2F;code&gt; has been automatically exported as a &lt;strong&gt;tool&lt;&#x2F;strong&gt;. But MCP not only defines tools, but also resources and resource templates. We support these with an automatic mapping in the following way:&lt;&#x2F;p&gt;
&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Agent&lt;&#x2F;th&gt;&lt;th&gt;Method&lt;&#x2F;th&gt;&lt;th&gt;MCP entity&lt;&#x2F;th&gt;&lt;&#x2F;tr&gt;&lt;&#x2F;thead&gt;&lt;tbody&gt;
&lt;tr&gt;&lt;td&gt;Singleton&lt;&#x2F;td&gt;&lt;td&gt;No parameters&lt;&#x2F;td&gt;&lt;td&gt;Resource&lt;&#x2F;td&gt;&lt;&#x2F;tr&gt;
&lt;tr&gt;&lt;td&gt;Non-singleton&lt;&#x2F;td&gt;&lt;td&gt;No parameters&lt;&#x2F;td&gt;&lt;td&gt;Resource template&lt;&#x2F;td&gt;&lt;&#x2F;tr&gt;
&lt;tr&gt;&lt;td&gt;Any&lt;&#x2F;td&gt;&lt;td&gt;Has parameters&lt;&#x2F;td&gt;&lt;td&gt;Tool&lt;&#x2F;td&gt;&lt;&#x2F;tr&gt;
&lt;&#x2F;tbody&gt;&lt;&#x2F;table&gt;
&lt;h3 id=&quot;metadata&quot;&gt;Metadata&lt;&#x2F;h3&gt;
&lt;p&gt;For every agent and agent method, we can attach a &lt;strong&gt;description&lt;&#x2F;strong&gt; and a &lt;strong&gt;prompt&lt;&#x2F;strong&gt;:&lt;&#x2F;p&gt;
&lt;div class=&quot;code-tabs&quot; id=&quot;codetabs-1&quot;&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;typescript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;@&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;description&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increments the counter by the number provided in the `n` parameter&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;@&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;prompt&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increment by a given number&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;async&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; increment_by&lt;&#x2F;span&gt;&lt;span&gt;(n: number): &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Promise&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span&gt;number&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt;  &#x2F;&#x2F; ...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[description(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increments the counter by the number provided in the `n` parameter&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[prompt(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increment by a given number&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; increment_by&lt;&#x2F;span&gt;&lt;span&gt;(&amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; n&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;@description(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increments the counter by the number provided in the `n` parameter&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;@prompt(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increment by a given number&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; incrementBy&lt;&#x2F;span&gt;&lt;span&gt;(n:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Int&lt;&#x2F;span&gt;&lt;span&gt;):&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Future&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Int&lt;&#x2F;span&gt;&lt;span&gt;]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;moonbit&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt;&#x2F;&#x2F;&#x2F;| Increments the counter by the number provided in the `n` parameter&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;#derive.prompt_hint&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;Increment by a given number&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Counter&lt;&#x2F;span&gt;&lt;span&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;increment&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;self&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Self&lt;&#x2F;span&gt;&lt;span&gt;, &lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;n&lt;&#x2F;span&gt;&lt;span&gt;: &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;UInt32&lt;&#x2F;span&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;-&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; UInt32&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;div&gt;
&lt;p&gt;Both are optional, and both are added to the MCP metadata.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;special-data-types&quot;&gt;Special data types&lt;&#x2F;h3&gt;
&lt;p&gt;It&#x27;s not strictly related to the MCP feature, and not even new in &lt;strong&gt;Golem 1.5&lt;&#x2F;strong&gt;, but the three special data types supported by all the Golem SDKs are a good match for exposing some special tools and resources through the MCP protocol:&lt;&#x2F;p&gt;
&lt;h4 id=&quot;unstructured-text&quot;&gt;Unstructured text&lt;&#x2F;h4&gt;
&lt;p&gt;Any method parameter or return type can be defined as &lt;strong&gt;unstructured text&lt;&#x2F;strong&gt;. Optionally a set of allowed &lt;strong&gt;language codes&lt;&#x2F;strong&gt; can be attached to the type:&lt;&#x2F;p&gt;
&lt;div class=&quot;code-tabs&quot; id=&quot;codetabs-2&quot;&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;typescript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;myMethod&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  anyText: UnstructuredText,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  constrainedText: UnstructuredText&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;#39;en&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;#39;de&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt;  &#x2F;&#x2F; ...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[derive(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;AllowedLanguages&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;enum&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyLangs&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; En&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt; #[code(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;de&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; German&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; my_method&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;self&lt;&#x2F;span&gt;&lt;span&gt;, &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    any_text&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; UnstructuredText&lt;&#x2F;span&gt;&lt;span&gt;, &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    constrained_text&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; UnstructuredText&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;MyLangs&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; myMethod&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  anyText:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextSegment&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;AllowedLanguages&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Any&lt;&#x2F;span&gt;&lt;span&gt;],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  constrainedText:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextSegment&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MyLangs&lt;&#x2F;span&gt;&lt;span&gt;]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;sealed trait&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyLangs&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyLangs&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; En&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyLangs&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  @golem.runtime.annotations.languageCode(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;de&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; German&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyLangs&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  implicit val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; allowed&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; AllowedLanguages&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MyLangs&lt;&#x2F;span&gt;&lt;span&gt;] =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    golem.runtime.macros.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;AllowedLanguagesDerivation&lt;&#x2F;span&gt;&lt;span&gt;.derived  &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;moonbit&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;#derive.text_languages&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;constrained_text&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;en&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;de&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyAgent&lt;&#x2F;span&gt;&lt;span&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;my_method&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  self&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  any_text&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;UnstructuredText&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  constrained_text&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;UnstructuredText&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;-&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Unit&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  ...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;div&gt;
&lt;h4 id=&quot;unstructured-binary&quot;&gt;Unstructured binary&lt;&#x2F;h4&gt;
&lt;p&gt;Similarly to &lt;strong&gt;unstructured text&lt;&#x2F;strong&gt;, we can also use &lt;strong&gt;unstructured binary&lt;&#x2F;strong&gt; parameters and return types, and optionally define the allowed &lt;em&gt;MIME types&lt;&#x2F;em&gt; for them:&lt;&#x2F;p&gt;
&lt;div class=&quot;code-tabs&quot; id=&quot;codetabs-3&quot;&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;typescript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;myMethod&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  anyBinary: UnstructuredBinary,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  image: UnstructuredBinary&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;#39;image&#x2F;png&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;#39;image&#x2F;jpeg&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt;  &#x2F;&#x2F; ...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[derive(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;Debug&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; Clone&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; AllowedMimeTypes&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;enum&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Image&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;    #[mime_type(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;png&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;    Png&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;    #[mime_type(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;jpeg&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;    Jpeg&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; my_method&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;self&lt;&#x2F;span&gt;&lt;span&gt;, &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    any_binary&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; UnstructuredBinary&lt;&#x2F;span&gt;&lt;span&gt;, &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    image&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; UnstructuredBinary&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;Image&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; myMethod&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  anyBinary:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; BinarySegment&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;AllowedMimeTypes&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Any&lt;&#x2F;span&gt;&lt;span&gt;],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  image:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; BinarySegment&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Image&lt;&#x2F;span&gt;&lt;span&gt;]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;sealed trait&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Image&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Image&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  @golem.runtime.annotations.mimeType(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;png&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Png&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Image&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  @golem.runtime.annotations.mimeType(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;jpeg&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Jpeg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Image&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  implicit val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; allowed&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; AllowedMimeTypes&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Image&lt;&#x2F;span&gt;&lt;span&gt;] =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    golem.runtime.macros.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;AllowedMimeTypesDerivation&lt;&#x2F;span&gt;&lt;span&gt;.derived&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;moonbit&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;#derive.mime_types&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;image&#x2F;png&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;image&#x2F;jpeg&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyAgent&lt;&#x2F;span&gt;&lt;span&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;my_method&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  self&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  any_binary&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;UnstructuredBinary&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  image&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;UnstructuredBinary&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;-&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Unit&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  ...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;div&gt;
&lt;h4 id=&quot;multimodal&quot;&gt;Multimodal&lt;&#x2F;h4&gt;
&lt;p&gt;Finally there is a special parameter type called &lt;strong&gt;multimodal&lt;&#x2F;strong&gt;, which is a special way to define methods (tools) that can work on multiple types of input. The default multimodal type just allows pasting either text or binary, but it is fully customizable with the above defined language and MIME type constraints, and can also include structured data.
By using multimodal types, and not just modelling the same input using custom data types, Golem can map these definitions better to MCP concepts.&lt;&#x2F;p&gt;
&lt;p&gt;The simplest version just accepts either an arbitrary text, or an arbitrary binary:&lt;&#x2F;p&gt;
&lt;div class=&quot;code-tabs&quot; id=&quot;codetabs-4&quot;&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;typescript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;textOrBinary&lt;&#x2F;span&gt;&lt;span&gt;(input: Multimodal) {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; ...&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; text_or_binary&lt;&#x2F;span&gt;&lt;span&gt;(&amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; input&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; Multimodal&lt;&#x2F;span&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; Multimodal&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; input&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; textOrBinary&lt;&#x2F;span&gt;&lt;span&gt;(input:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MultimodalItems&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Basic&lt;&#x2F;span&gt;&lt;span&gt;):&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Future&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MultimodalItems&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Basic&lt;&#x2F;span&gt;&lt;span&gt;]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;moonbit&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyAgent&lt;&#x2F;span&gt;&lt;span&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;text_or_binary&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  self&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  input&lt;&#x2F;span&gt;&lt;span&gt; : @types.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Multimodal&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextOrBinary&lt;&#x2F;span&gt;&lt;span&gt;],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;-&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; String&lt;&#x2F;span&gt;&lt;span&gt; { ... }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;div&gt;
&lt;p&gt;We can add a third option in the form of a structured data type to this:&lt;&#x2F;p&gt;
&lt;div class=&quot;code-tabs&quot; id=&quot;codetabs-5&quot;&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;typescript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;type&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyStructuredType&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; ...&lt;&#x2F;span&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;textOrBinaryOrStructured&lt;&#x2F;span&gt;&lt;span&gt;(input&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MultimodalCustom&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;) {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; ...&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[derive(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;Schema&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;struct&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt; &#x2F;* ... *&#x2F;&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; text_or_binary_or_structured&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    input&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; MultimodalCustom&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; MultimodalCustom&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; input&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;final case class&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt;&#x2F;* ... *&#x2F;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; implicit val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; schema&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Schema&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;] =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Schema&lt;&#x2F;span&gt;&lt;span&gt;.derived }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; textOrBinaryOrStructured&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  input:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MultimodalItems&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;WithCustom&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;):&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Future&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MultimodalItems&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;WithCustom&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;]]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;moonbit&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;#derive.golem_schema&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;all&lt;&#x2F;span&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;struct&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt; { &#x2F;* ... *&#x2F; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyAgent&lt;&#x2F;span&gt;&lt;span&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;text_or_binary_or_structured&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  self&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  input&lt;&#x2F;span&gt;&lt;span&gt; : @types.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Multimodal&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;CustomModality&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;MyStructuredType&lt;&#x2F;span&gt;&lt;span&gt;]],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;-&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; String&lt;&#x2F;span&gt;&lt;span&gt; { ... }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;div&gt;
&lt;p&gt;Or we can fully customize the multimodal behavior by defining our own variant type it maps to:&lt;&#x2F;p&gt;
&lt;div class=&quot;code-tabs&quot; id=&quot;codetabs-6&quot;&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;typescript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;export type&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextOrImage&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;  |&lt;&#x2F;span&gt;&lt;span&gt; { tag&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;#39;text&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;; val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; UnstructuredText&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;#39;en&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;#39;de&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;] }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;  |&lt;&#x2F;span&gt;&lt;span&gt; { tag&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;#39;image&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;; val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; UnstructuredBinary&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;#39;image&#x2F;jpeg&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;#39;image&#x2F;png&amp;#39;&lt;&#x2F;span&gt;&lt;span&gt;]&amp;gt; };&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;fullyCustom&lt;&#x2F;span&gt;&lt;span&gt;(input&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MultimodalAdvanced&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;) {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; ...&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[derive(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;AllowedLanguages&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;enum&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextLang&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; En&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt; #[code(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;de&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; German&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[derive(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;AllowedMimeTypes&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;enum&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; ImageType&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;    #[mime_type(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;jpeg&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; Jpeg&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;    #[mime_type(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;png&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; Png&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #986801;&quot;&gt;#[derive(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;Schema&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; MultimodalSchema&lt;&#x2F;span&gt;&lt;span style=&quot;color: #986801;&quot;&gt;)]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;enum&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextOrImage&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;    Text&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;UnstructuredText&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;TextLang&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;),&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;    Image&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;UnstructuredBinary&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;ImageType&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;),&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; fully_custom&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;    input&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; MultimodalAdvanced&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt; MultimodalAdvanced&lt;&#x2F;span&gt;&lt;span&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0184BC;&quot;&gt;TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; input&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;sealed trait&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextLang&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextLang&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  @golem.runtime.annotations.languageCode(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;en&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; En&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextLang&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  @golem.runtime.annotations.languageCode(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;de&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; De&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextLang&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  implicit val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; allowed&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; AllowedLanguages&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextLang&lt;&#x2F;span&gt;&lt;span&gt;] =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    golem.runtime.macros.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;AllowedLanguagesDerivation&lt;&#x2F;span&gt;&lt;span&gt;.derived&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;sealed trait&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; ImageType&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; ImageType&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  @golem.runtime.annotations.mimeType(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;jpeg&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Jpeg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; ImageType&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  @golem.runtime.annotations.mimeType(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;image&#x2F;png&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  case object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Png&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; extends&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; ImageType&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;  implicit val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; allowed&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; AllowedMimeTypes&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;ImageType&lt;&#x2F;span&gt;&lt;span&gt;] =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    golem.runtime.macros.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;AllowedMimeTypesDerivation&lt;&#x2F;span&gt;&lt;span&gt;.derived&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;final case class&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  text:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextSegment&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextLang&lt;&#x2F;span&gt;&lt;span&gt;],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  image:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; BinarySegment&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;ImageType&lt;&#x2F;span&gt;&lt;span&gt;],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;object&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextOrImage&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt; implicit val&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt; schema&lt;&#x2F;span&gt;&lt;span&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; GolemSchema&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;] =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #A0A1A7;font-style: italic;&quot;&gt; &#x2F;* derived *&#x2F;&lt;&#x2F;span&gt;&lt;span&gt; }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt; fullyCustom&lt;&#x2F;span&gt;&lt;span&gt;(input:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Multimodal&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;]):&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; Future&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Multimodal&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;]]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;pre class=&quot;giallo&quot; style=&quot;color: #383A42; background-color: #FAFAFA;&quot;&gt;&lt;code data-lang=&quot;moonbit&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;#derive.multimodal&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;all&lt;&#x2F;span&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;enum&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; TextOrImage&lt;&#x2F;span&gt;&lt;span&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;  Text&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;UnstructuredText&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;  Image&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;UnstructuredBinary&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;#derive.text_languages&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;input.Text&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;en&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;de&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;#derive.mime_types&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt;&amp;quot;input.Image&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;image&#x2F;jpeg&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #50A14F;&quot;&gt; &amp;quot;image&#x2F;png&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;pub fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; MyAgent&lt;&#x2F;span&gt;&lt;span&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #4078F2;&quot;&gt;fully_custom&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  self&lt;&#x2F;span&gt;&lt;span&gt; : &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Self&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #E45649;&quot;&gt;  input&lt;&#x2F;span&gt;&lt;span&gt; : @types.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;Multimodal&lt;&#x2F;span&gt;&lt;span&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt;TextOrImage&lt;&#x2F;span&gt;&lt;span&gt;],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;) &lt;&#x2F;span&gt;&lt;span style=&quot;color: #A626A4;&quot;&gt;-&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C18401;&quot;&gt; String&lt;&#x2F;span&gt;&lt;span&gt; { ... }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;div&gt;
&lt;h4 id=&quot;remarks&quot;&gt;Remarks&lt;&#x2F;h4&gt;
&lt;p&gt;None of these special data types are MCP specific - using them in our agent code is not constraining them to be only called through MCP, they can still be invoked through agent-to-agent communication, mapped to HTTP APIs and so on.&lt;&#x2F;p&gt;
</content>
        
    </entry>
</feed>
