Skip to main content

Shinkai DSL

The shinkai_dsl is a domain-specific language (DSL) designed to define workflows for agents in the Shinkai project. It allows users to specify sequences of steps and operations that an agent should perform. Here's a breakdown of its components and how it works:

Components

  1. Workflow: The top-level structure that contains multiple steps.
  2. Step: A unit of work within a workflow, containing a body of operations.
  3. Step Body: Contains conditions, register operations, actions, and loops.
  4. Registers: Variables prefixed with $R used to store intermediate results.
  5. External Function Calls: Invocations of functions defined outside the DSL.

Currently, the syntax is still under heavy development but we expect to be ready for early testing in the next few weeks. We are also working towards an integrated playground into the Shinkai App to allow you to easy test your DSLs. Later on, we are also planning to implement a GUI so users won't even need to write the DSL, but rather they will be able to use the GUI to build their workflows. And of course, the end goal is that you could explain what you want to a local AI and it will auto-generate the GUI / DSL for you, although we will always expect that the user will have to validate the generated DSL before it can be used.

Example of DSL

 workflow WebProcess v0.1 {
step Initialize {
$PROMPT = "You are an expert summarized. Summarize the following webpage content and output the result in a markdown format: "
}
step Download {
$CONTENT = call download_webpage_and_save_as_markdown($INPUT)
}
step Summarize {
$RESULT = call inference($PROMPT, $CONTENT)
}
}

DSL Specification

The DSL is defined by the following grammar:

workflow  = { "workflow" ~ identifier ~ version ~ "{" ~ step+ ~ "}" }
step = { "step" ~ identifier ~ "{" ~ step_body ~ "}" }
step_body = { (condition | register_operation | action | for_loop)+ }
condition = { "if" ~ expression ~ "{" ~ step_body ~ "}" }
for_loop = { "for" ~ identifier ~ "in" ~ (split_expression | range_expression) ~ "{" ~ step_body ~ "}" }
action = { external_fn_call | command ~ "(" ~ (param ~ ("," ~ param)*)? ~ ")" }
command = { identifier }
param = { string | number | boolean | identifier | register }
register = { "$" ~ "R" ~ ASCII_DIGIT+ }
// New rule for registers
external_fn_call = { "call" ~ identifier ~ "(" ~ (param ~ ("," ~ param)*)? ~ ")" }
expression = { range_expression | simple_expression ~ (comparison_operator ~ simple_expression)? }
simple_expression = { identifier | number | boolean | string | register }
range_expression = { identifier ~ ".." ~ identifier }
register_operation = { register ~ "=" ~ (external_fn_call | value) }
// New rule for register operations
comparison_operator = { "==" | "!=" | ">" | "<" | ">=" | "<=" }
value = { string | number | boolean | identifier | register }
split_expression = { (register | identifier | string) ~ ".split(" ~ delimiter ~ ")" }
version = { "v" ~ ASCII_DIGIT+ ~ ("." ~ ASCII_DIGIT+)* }
identifier = @{ (ASCII_ALPHANUMERIC | "_")+ }
string = _{ "\"" ~ (!"\"" ~ ANY)* ~ "\"" }
delimiter = { "\"" ~ (!"\"" ~ ANY)* ~ "\"" }
number = _{ ASCII_DIGIT+ }
boolean = { "true" | "false" }
WHITESPACE = _{ " " | "\t" | "\n" | "\r" }

The goal is to keep the DSL as simple as possible while still being able to express the most common use cases. We are still working on the DSL and we will keep adding new features to it.