Extending the interpreter with new tags
=======================================

The interpreter `pyaiml21` was designed with the idea of creating
new (AIML) tags and enabling the interpreter and parser to properly
handle them. These extensions can be applied to pattern paths or
the template elements, never to the structure of the category.

As there is a significant difference between tags or words
constituting the pattern path (found inside `<pattern>`, `<that>`
and `<topic> within the category element) and elements within the `<template>`
tag, there is also a difference in writing the extensions for them.



Pattern extensions
------------------

To create an extension of a pattern path, you will need to create an
object of type `Pattern` with the required attributes.

Consider that we would like to allow the interpreter to properly parse
and evaluate a non-standard `<regex>` tag that contains a text with
the regex that matches any single word with this regex.

The implementation of the individual attributes of the created tag might
go as follows:

.. code-block:: python

    PRIORITY = 100  # check priorities in /std/_pattern.py
    IS_STAR = True
    REGEX_KEY = "__REGEX__"

    def recognise_regex(node: Node) -> bool:
        return node.tag == "regex"

    def parse_regex(node: Node, logger: Logger) -> None:
        assert recognise_regex(node)
        # regex should have a text child
        if not node.children or not node.children[0].is_text_node:
            logger.error(node, "regex expecting text element")

    def advance_regex(tree_node: GMNode, node: Node) -> GMNode:
        regex = node.children[0].text
        return tree_node{REGEX_KEY][regex]

    def match(seq, pos, node: GMNode, gm):
        if REGEX_KEY not in node.children:
            return None  # no regex child
        regex_node = node.children[REGEX_KEY]
        for regex, node in regex_node.children.items():
            if re.match(regex, seq[pos]):
                yield node

Then to create the actual object, you can use:

.. code-block:: python

    REGEX_NODE = Pattern(PRIORITY, IS_STAR, parse_regex,
                         recognise_regex, advance_regex,
                         match)

and to add it to the chatbot's knowledge:

.. code-block:: python

    bot.add_pattern_tag(REGEX_NODE)

From this momment on, the chatbot will be able to parse and
evaluate any match including the regexes.



Template Extension
------------------

Creation of a <template> side tag is even simpler, you need
to defined just the name of the tag, how to parse it and how
to evaluate the `Node` with this tag.

Suppose you want define a tag `<python>` that takes no arguments
and runs arbitrary python code (*this is quite dangerous, do it only
at your own risk*) and returns whatever the code prints to stdout:

.. code-block:: python

    from io import StringIO
    from contextlib import redirect_stdout


    TAG = "python"

    def parse_python(node: Node, logger: Logger) -> None:
        # nothing to check here, we are not expecting any attributes
        pass

    def eval_python(walker, node: Node) -> str:
        the_code = walker.eval_seq(node.children)
        output_stream = StringIO()
        with redirect_stdout(output_stream):
            exec(the_code)
        return output_stream.getvalue()

Then to let the chatbot know about this tag, you can use:

.. code-block:: python

    bot.add_template_tag(TAG, parse_python, eval_python)



Other examples
--------------

For more examples, checkout the implementation of the AIML standard
tags that is inside `/stdlib/` directory.