AI Command

3.1.1 AI

DESCRIPTION

AI - interact with the AI (since 8.2).

“AI chat” sends a message to the AI and saves the response to the _AI_RESPONSE script variable. To attach the current desktop screen image and/or text file(s) use the screen and attach parameters.

The AI response can be tested in three ways:

• Use the “text” and optionally the “distance” parameters to verify that the AI response contains a certain word or phrase. The script will be terminated if the response doesn’t match.

• For more flexible text matching search the response for a match of “pattern” regular expression. The script will be terminated if the response doesn’t match.

• If neither “text” nor “pattern” is specified the response will be saved to the _AI_RESPONSE script variable and the script will be allowed to continue. You may subsequently test the variable using if/else together with the boolean expressions such as "contains", "startswith", "endswith" or "matches". Another option is to parse the text with the String command.
  See the Examples section for more.
  

SYNOPSIS

ai chat [message=<text>] [ai=<Claude|OpenAI>] [screen=<true|false>]  [attach=<file(s)>]  [text=<pattern>]  [distance=<number>] [pattern=<regexp>]

• Red colour indicates obligatory parameters

OPTIONS

message=<text>

• The message text (“question”), for example “Is there a dog on the screen?”.

ai=<Claude|OpenAI>

• An optional AI name, either “Claude” or “OpenAI”. The name is typically specified only when multiple AIs are configured and you wish to target a particular one. If the name is not specified then the command uses the AI that is configured or Claude if both are configured.

screen=<true|false>

• The value of “true” will attach a copy of the connected desktop image to the AI question (optional). The default value is “false” (do not attach the screen).

attach=<file(s)>

• Optional file or a colon or semicolon separated list of files to attach to the question. This is limited to:

  • Image files supported by Java (PNG, JPG, BMP, GIF, TIFF, WBMP)

  • Plain text files and files with readable text content such as XML, HTML and similar.

[text=<pattern>]

• Optional case sensitive text to match the AI response against (“contains”). For example, the value of “dog” will produce a pass result when the AI response contains the word of “dog” but it will fail if it contains “DOG” or “Dog”. For tolerant matching use the “distance” or “pattern” parameter. The “text” and “pattern” parameters are mutually exclusive and only one can be used at a time.

[distance=<number>]

• Optional distance used in conjunction with the "text" parameter to perform tolerant text search. When the parameter is set to the default value of 0 (no distance) the command falls back to plain string search where the argument text is searched for the first occurrence of the provided string. 

  The tolerant (fuzzy) text search is performed for the distance values of 1 and higher. The text is searched for an occurrence of a string which is similar enough to the specified one. The tolerance (similarity) is based on the Levenshtein distance. It is defined as the minimum number of edits needed to transform one string into the other, with the allowable edit operations being insertion, deletion, or substitution of a single character. The distance approximately specifies how many characters may be omitted or not recognized properly at a maximum to consider the sample text equivalent. 
  For example, the response of “There is fog” will match for the parameters of “text=dog distance=1” because a substitution of one character in “dog” produces “fog”.

[pattern=<regexp>]

• Search the response for a java.util.regex.Pattern compliant regular expression (optional). The regular expression of "." matches by default any character except for the line terminator. This behaviour may be changed through command preferences. 
  The “text” and “pattern” parameters are mutually exclusive and only one can be used at a time.

EXAMPLES

• Verify that there is a dog on the screen and terminate the script if not.

Ai chat message="Does the screen show a dog? Answer just \"yes\" or \"no\"." screen=true ai=OpenAI pattern=[Yy]es

• Verify that there is a dog on the screen and then act based on the result

Ai chat message="Does the screen show a dog? Answer just \"yes\" or \"no\"." screen=true ai=OpenAI
if ("{_AI_RESPONSE}" matches "[Yy]es") {
    // There's a dog on the screen -> do something
    ...
} else {
    // There's no dog -> do something else
    ...
}