-
Notifications
You must be signed in to change notification settings - Fork 0
Tutorial
The old Readme page began to be quite... loooong... So I decided to write this tutorial as a quick start to the most wonderful template / bootstrap tool. By the way, it could also be used for code generation...
(This tutorial is written for linux / bash... Adapt the shell commands for other systems)
First, create a sandbox directory :
mkdir sandbox
cd sandbox
If you just installed skbs, run the following to initialize the config and install the default provided templates :
skbs create-config
skbs install-defaults
Any file is actually a template...
For example run the following:
echo "Hello world, this is me life should be
Fun for everyone" > my_first_template.txt
Then you can "execute" it (big word to mean "copy" in our example...) with :
skbs gen my_first_template.txt result.txt --
The double dash at the end is optional... It's meant to separate skbs arguments from template arguments.
If you open result.txt, you will see it's content is the same as my_first_template.txt
...But, there is cp to do that ?
Yes. But with skbs, you can install/uninstall this template to have access to it everywhere.
Type :
skbs install my_first_template.txt
To list the installed templates, run the following :
skbs list
You should have the following output:
User-installed templates :
my_first_template.txt
Default templates :
skbs
skbs.sft
You can recall any of these templates by prefixing them with @, no matter of the current working directory:
skbs gen @my_first_template.txt result2.txt
...And again another copy of this file.
If you change ./my_first_template.txt, it won't affect the installed version. To be able to edit the template and automatically reflect the change, you can use instead a symlink install (that will use instead a symbolic link):
skbs install my_first_template.txt --symlink
You can also change the name used once installed using the -n option (see skbs install --help for more details).
Installing as a symlink is the recommended way, specially if your template is in a versioned (with git or other).
Copying is not that exciting. What we would like to do is being able to pass parameters to our templates, and then, change the content of the generated file depending on the context, or user provided arguments.
But first, the Dynamic part.
The previous template we created (a simple file...) is what we call a "self-contained single file template". Such files have three types of "block":
- Raw blocks (output as is)
- Code blocks
- Code expression blocks (which are embedded in a raw block)
Raw blocks are everything that is not a python bock or expression. They are output as they are.
A Code block is a set of contiguous lines starting with a "statement line start" prefix (which is ## by default). Code blocks are python code except that instead of the indent to delimit scope, we use a line containing a single -.
In detail, a line endings with :, increment the indent level for the following ones, A line containing a single - decrements it.
Example:
echo "## a = 'world'
## b = 42
## if b == 42:
## c = 5
## -
## else:
## c = 6
## -" > my_first_code_template.txt
...But this template will actually create an empty file...
Codes and raw blocks can be intermixed (and this is where it starts to get interesting...)
Try this template:
echo "## for i in range(5):
Hello world, this me [...]
## -" > my_first_INtrEsTinG_code_template.txt
skbs gen my_first_INtrEsTinG_code_template.txt INtrEsTinG_result.txt
cat INtrEsTinG_result.txt
See ? "Hello world, this me [...]" is repeated 5 times.
Raw blocks actually behave like a python statement : each time it is reached, it is printed.
Here, the for loops five times, and each time encounter the raw block, thus it gets printed the same amount of times.
It works with all python constructs (while, if, else, try..except etc.).
Code blocks are nice to alter the flow, but what about putting dynamically computed value into the result ?
The result of a python expression can be injected into a raw block. To do so, you need to surround it with the "begin expression" prefix ({{ by default) and the "end expression" suffix (}} by default). Everything that would be accessible at the expression position in a code block is in the expression.
Example:
echo "## for i in range(5):
{{i}}) Hello world, this me [...]
## -" > my_first_very_INtrEsTinG_code_template.txt
skbs gen my_first_very_INtrEsTinG_code_template.txt very_INtrEsTinG_result.txt
cat very_INtrEsTinG_result.txt
(Internally, the raw block is indeed compiled as a statement, and uses .format() to inject the values)
What if I want to write a latex template ? There are {{ everywhere... And ## messes up with the highlighting
Yup, true story...
Good news: you can change the "statement line start", "begin expression" and "end expression" tokens. Simply write a "Syntax header line" ("synheader line") like this at the top of your template file:
<statement line start> # <begin expression>__skbs_template__<end expression>
Example:
echo '%% # <<__skbs_template__>>
%% for i in range(5):
<<i>>) Some complex math : $ \frac{{(5-3)}^{2+3}}{\pi} $
%% -' > latex_powa.txt
skbs gen latex_powa.txt latex_result.txt
cat latex_result.txt
...It is a good practice to always provide this header, even if using the default syntax, because you know at first sight it's a template. Also, it permits to skbs to know if a file is a template or a raw file.
Fun fact : This header line was designed such that it's a comment once compiled, thus not affecting the result
As discussed earlier, template themselves can accept arguments. They are stored in the built-in args global variable.
For example, to accept key=value pairs, you can do the following:
echo '## # {{__skbs_template__}}
## d = { k.strip(): v.strip() for a in args for k, v in (a.split('"'"'='"'"'),) }
Your name is {{d.get("name", "I don'"'"'t know")}}
You are {{d.get("age", "(I don'"'"'t know)")}}
' > you.txt
skbs gen you.txt me.txt -- age=23
cat me.txt
(The '"'"' expands to ' after bash processed it... Open you.txt for a more readable version...)
...But a more versatile solution is to use click: https://click.palletsprojects.com/en/8.0.x/
Skbs has first-class support with this library. The default skbs template even provide a --click/-c option to add the click boilerplate.
It is always a good practice to use the default templates to create new ones:
skbs gen @skbs.sft my_cool_template.txt -- --click
This will create the following file :
## # {{__skbs_template__}}
## __doc__ = """
## Template my_cool_template.txt
## """
## _p = C()
##
## @click.command(help=__doc__)
## def main(**kwargs):
## _p.update(kwargs)
## -
##
## with click.Context(main) as ctx:
## __doc__ = main.get_help(ctx)
## -
##
## if ask_help :
## raise EndOfPlugin()
## -
##
## invokeCmd(main, args)
Let's explain it line by line:
## # {{__skbs_template__}}
... Declare the tempiny syntax (the default one). This can be changed with the --syn <statement line start> <begin expression> <end expression template option.
## _p = C()
C() creates a dict-like that maps attributes to its items (accessible with []).
_p Is the built-in name "plugin variables". Here, since it's a single file template, it's not defined, thus we match the name here. (Note: single file templates were added afterward in the development process. Later in this tutorial, we will talk about directory-based template, and this will make sense)
## @click.command(help=__doc__)
## def main(**kwargs):
## _p.update(kwargs)
## -
This is the click main command (it could be a group by the way). By default, it injects all options and argument into _p.
See click's documentation https://click.palletsprojects.com/en/8.0.x/#documentation for more details on how to write a CLI.
## with click.Context(main) as ctx:
## __doc__ = main.get_help(ctx)
## -
##
## if ask_help :
## raise EndOfPlugin()
## -
##
## invokeCmd(main, args)
This is some click boilerplate to retrieve the documentation. ask_help is True if the user invoked the template with @help as destination, or passed a --help argument to the template.
Raising EndOfPlugin is the way to stop and say "we are done". If the user want the help, it's better to not output anything, and just stop here. Note that even if there are raw blocks before, no file will be written (it's buffered, and raising this exception won't write the buffer).
Then invokeCmd(main, args) invoke the main commands with the built-in args containing the template user-provided arguments. This function cares about all the details to invoke click redirecting click.echo and not quitting. This also let you a chance to alter the arguments before passing them to click.
To display the help, a __doc__ or alternatively (but deprecated) help global variable must be defined containing what will be sent to stderr.
It's nice to have all these tool to create wonderful templates... But it's still only one file...
What about a template that could spawn a full file tree ?
To create such a template, you need 3 things:
- A directory named as the template
- Inside it, a directory named "root", that will serve to contain what the template will spawn
- ...And a "plugin.py" file, that acts as an entry point for configuration, and interpreting the command line arguments
Like this:
my_template/
├─ root
│ └─ ... (content of the template)
└─ plugin.py
To understand better, let's generate the bootstrap for this kind of template
skbs gen @skbs my_first_dir_template -- --click
Here is the content of plugin.py :
__doc__ = """
Template my_first_dir_template
"""
try:
inside_skbs_plugin
except:
from skbs.pluginutils import IsNotAModuleOrScriptError
raise IsNotAModuleOrScriptError()
plugin = C()
@click.command(help=__doc__)
def main(**kwargs):
plugin.update(kwargs)
with click.Context(main) as ctx:
__doc__ = main.get_help(ctx)
if ask_help :
raise EndOfPlugin()
invokeCmd(main, args)
If you recall, it's almost exactly the same as the single file template header. The only difference is that it is raw python (no need for code blocks etc. since this file is only an entry point).
The plugin variable will be available in all template files as a global variable. Note that it is also aliased to the shorter _p to avoid typing too much...
The root directory represents the destination you specify when you invoke the template. Note that no file in the destination will be removed, they can only be overwritten by a file generated from the template.
Try this :
cd my_first_dir_template/root
echo "A very useful file" > dunnot_d3l3t3.txt
cd ../..
skbs gen my_first_dir_template my_first_output
Notice this time, my_first_output is a directory, and the file we created inside root is copied into it.
Files inside root can also be template, just add the template syntax header line !
For example, let's cpp class template :
First, create the file tree :
cpp/class
├─ root
│ ├─ class.h
│ └─ class.cpp
└─ plugin.py
skbs gen @skbs cpp/class -- --click
touch cpp/class/root/class.h
touch cpp/class/root/class.cpp
Now, first, let's accept a class name as template argument, an optional author option, and also optional base classes. This is the new plugin.py :
__doc__ = """
Template cpp/class.
Creates a cpp class with NAME.h and NAME.cpp
"""
try:
inside_skbs_plugin
except:
from skbs.pluginutils import IsNotAModuleOrScriptError
raise IsNotAModuleOrScriptError()
plugin = C()
@click.command(help=__doc__)
@click.option('--author', '-a', type=str, default=None, help="Author's name") # The author option. If no author is passed, None is the default value
@click.option('--base', '-b', type=(str, str), multiple=True) # The base option, you can pass it multiple time for multiple inheritance
@click.argument('name') # The name argument
def main(**kwargs):
plugin.update(kwargs)
with click.Context(main) as ctx:
__doc__ = main.get_help(ctx)
if ask_help :
raise EndOfPlugin()
invokeCmd(main, args)
Then, class.h and class.cpp.
Uh oh, the files are named "class.cpp", but we are almost sure, the user won't name its class "Class" (moreover, it's quite close from the "class" keyword...)
But no worry, skbs support dynamic filename...
Let's continue the cpp/class template we started in the previous section.
As stated, you can dynamically change a template file name, and it's quite trivial I don't know why I dedicate a section to this (spoiler alert : just for it to appear in the table of content...).
In a template file, you can declare a new_path variable, that is either a str or a pathlib.Path. The file will be put at this place, relative to the template root directory (the destination the user supply when invoking the template.)
Another variable is available : dest holds the original output path (relative to root).
The recommended way to change the name of the current file in the output is to do as follows :
new_path = dest.with_name('new_name')
This way works across subdirectories.
Here are a real application with or cpp/class template :
class.h:
//# # {{__skbs_template__}}
//# hguard = _p.name.upper() + '_H'
//# new_path = dest.with_name(_p.name.lower() + '.h')
//# name = _p.name
//# if _p.author is not None:
/*
* Author : {{_p.author}}
*/
//# -
#ifndef {{hguard}}
#define {{hguard}}
//# if len(_p.base) :
class {{name}} : {{', '.join( (scope + ' ' + base) for scope, base in _p.base )}}{
//# -
//# else :
class {{name}}{
//# -
public:
{{name}}();
~{{name}}();
};
#endif // {{hguard}}
class.cpp:
//# # {{__skbs_template__}}
//# from skbs.pluginutils import looptools
//# hguard = _p.name.upper() + '_H'
//# new_path = dest.with_name(_p.name.lower() + '.cpp')
//# name = _p.name
//# if _p.author is not None:
/*
* Author : {{_p.author}}
*/
//# -
#include "{{name.lower()+".h"}}"
//# if len(_p.base) :
{{name}}::{{name}}() :
//# for is_last, (scope, base) in looptools.check_last(_p.base):
{{base}}(){{'{' if is_last else ','}}
//# -
//# -
//# else:
{{name}}::{{name}}(){
//# -
}
{{name}}::~{{name}}(){
}
And what about a directory ?
This one uses an "obscure" feature from before version 2...
You can have a file _template. inside a directory. This file is a python file, and anything possible in a regular template file, you can do in it too, and it will affect the directory (and all its content).
For example, you can set new_path, and the whole directory will be put at new_path
Don't bother about the name for now, some explanation can be found in Advanced and Obscure features
What if the destination of the template contains a file that you want to keep, but would be overwritten by the template ?
A template can flag files and directory as "optional". There are two ways.
- Declare and set the
is_opttoTrue. Note that this way is not possible for raw files (without the syntax header line, copied as they are) - Add a
_opt.prefix to the file name, works everywhere.
The is_opt is the preferred way, because you can make it dynamic. For example, you can add a flag in the command-line to ask the user if (s)he want to overwrite.
Sometimes, you could want to provide some optional features to your template, and output a file or not depending on an option. Another use case is if a same file system node that could be either a file either a directory depending on the context.
To prevent a file / directory to be output, you can use the exclude() function. Calling it will stop evaluating the current file and will tell to skbs to skip it / exclude it from the output.
For the second use case, a common solution is to name the file differently. If the file should be output, the directory's _template. should call exclude, and the file assign new_path to the directory's name. In the opposite case, the file should call exclude().
Look at the skbs template source for a good example.
You already had quite an overview of skbs !
Ready to unleash the full power ?
Let's attack sections and placeholder, what truly makes skbs superior to other template/bootstrap software : The ability to do partial overwriting !
Placeholders (and sections) are some kind of "marker" that you can put into your template (don't worry, it's just python functions you call, no new syntax), to deal with existing file.
To better understand how they work, you should first imagine the template file is run a first time and output to what we will call a "virtual output", and that without the section stuff, this virtual output would overwrite what we will call the "original file".
Depending on what you want to achieve, there are 2 modes of operation : use the original file, but replace the content of the section by the ones from the virtual output, or use the virtual output and replace the section content by the ones from the original file.
Here is an example to illustrate it :
Original file │ Virtual output │ Result in │ Result in
│ (suppose a template │ keep_only_sections= │ keep_only_sections=
│ file that has been │ False │ True
│ processed) │ mode (default) │ mode
───────────────┼─────────────────────┼─────────────────────┼─────────────────────
foo │ oof │ foo │ oof
BEGIN SECTION │ BEGIN SECTION │ BEGIN SECTION │ BEGIN SECTION
AAA │ BBB │ BBB │ AAA
END SECTION │ END SECTION │ END SECTION │ END SECTION
bar │ arb │ bar │ arb
│ │ │
Of course, the beginning of a section is not necessarily "BEGIN SECTION" ! This is only an example, but in this example, it is indeed the string that we want to match.
To define a section, there are two python functions : beginSection() and endSection()
The template that would produce the previous result would look like this one :
## # {{__skbs_template__}}
## __doc__ = """
## Template example_sections
## """
##
## if ask_help :
## raise EndOfPlugin()
## -
## keep_only_sections = len(args) == 1 and args[0] == 'yes'
##
oof
## beginSection()
BEGIN SECTION
BBB
END SECTION
## endSection()
arb
By default, beginSection() matches its next line, and endSection() its previous line.
The number and the location of line to match is set with the m and n argument to beginSection, endSection and also placeholder that we will see just after.
m is the first line to match relative to what we want to define, and n is the "past-the-end" line.
For example to match 2 lines before and one after, set m = -2 and n = 1.
Internally, the algorithm knows where are the section from the virtual output thanks to the beginSection()/endSection() calls.
Then, for each line from the original file, it checks for the first section which beginning matches the line. Then it steps lines in the file until the end of the section matches. Once the beginning and the end of the section matched, the section is popped to avoid matching again. If the end of file is reached without finding the end of the section, it tries the next one.
Once The end of file is reached, the replacements are made according to the rules above.
This way, sections can be dynamically created, and matched in an intuitive way, even across for loops... The only limitation as of v2.1 is that sections cannot be nested (They will in some future versions).
beginSection and endSection also accept a cb callback parameter. It should be a function cb(lines:list[str], i:int) -> bool where lines is a list of all the lines in the original file, and i the index of the current line.
This callback should return True if lines[i] is the first line in section (when passed to beginSection) or the line just after the last line of the section (for endSection).
beginSection also accepts an overwrite parameter. When set to True, the content from the original file's section will always be used, and when False, the content from virtual output will. If left unset or None, it behaves like the previous example.
It can also be to a callable object with this signature : overwrite(original: list[str], virtual: list[str], ctx: Callable[[], 'Context']) -> list[str]. Where origninal and virtual are the content of the matched sections, and ctx is a callable to obtain the context. It returns an object with the following attributes (read-only):
-
o: Original file :-
o.lines: original file lines -
o.sec_b: begin of section -
o.sec_e: end of section
-
-
vVirtual output :-
o.lines: original file lines -
o.sec_b: begin of section -
o.sec_e: end of section
-
keep_only_sections
Now, what happens when a section is not matched in the original file ?
Here comes the placeholder.
A placeholder behave like beginSection (without the overwrite parameter), and take a name parameter to identify it.
It is then possible to supply a placeholder argument to beginSection with the placeholder name as value. If the section is not matched in the original file, but the placeholder referenced is, then the section is put just after the placeholder.
Note : since the section definitions are in the template, it is not possible to have a section in the original file not matched in the virtual input. Either a section matched in the original file either not. Thus, the placeholders are meaningless if keep_only_section is True.
Though, in such case, no error will be raised so that no additional logic is required to toggle keep_only_sections.