z advanced Topics

File and Second Streams

Typically, data is piped into pyp just like any other unix utility through STD-IN. There are, however, two other completely independent ways to get data into pyp. These other streams can then be manipulated separately as well as combined in various ways with the main stream.

File inputs can be specified using the --text_file flag, and are referred to on a line-by line-basis using the variable 'fp'. For example, to print out the std-in line-by-line next to each line in a file use this:

`pyp "p + fp" --text_file example.txt`

Second stream inputs, on the other hand, are anything after the pyp command that is not associated with an option flag. This can then be accessed separately from the primary stream by using the variable 'sp'. To print out the std-in line-by-line next to each sp_example, use this:

`pyp "p + sp" sp_example1 sp_example2 sp_example3`

File and Second Stream List Variables

List operations can now be performed on file inputs and second stream inputs using the variables spp and fpp, respectively. These essentially treat each input as a list, so you can use standard python list methods as well as any of the specialized pyp list methods. Once modified, these changes will "stick", so future references to the list will use the new list.

For example to sort a file input use: ```

fpp.sort()

``` Once this operation takes place, the sorted fpp will be used for all future operations, such as referring to the file input line-by-line using fp.

If you need arbitrary strings added to your list, you can just use simple python list additions:

`fpp + ['last list']`

You can also add these inputs to the std-in stream using like this:

`pp+fpp`

If pp is 10 lines, and fpp is 10 line, this will result in a new pp stream of 20 lines, with the first 10 being from pp and the last 10 being from fpp. fpp will remain untouched; only pp will change with this operation.

Of course, you can trim these to your needs using standard python list selection techniques: pp[0:5]+ fpp[0:5]

This will result in a new composite input stream of 10 lines.

Keep in mind that the length of fpp and spp is trimmed to reflect that of std-in. If you need to see more of your file second stream input, you can extend your std-in stream simply:

`pp+['']*10`

will add 10 blank lines to std-in, and thus reveal another 10 lines of fpp if available.

Also, there are a few useful python math functions that work on lists of integers or floats like sum, min, and max. For example, to add up all of the integers in the last column input: ```

whitespace[-1] | int(p) | sum(pp)

Other Features
`p.ext` string attribute added for file extension
`p.letters()`|returns array of contiguous letters in p
`p.digits()` |returns array of contiguous numbers in p.| 
`p.punctuation()`

The next functions are just some nice cleanup tools:

`p.kill(STR1, STR2,...)` takes multiple arguments. all STR* will be removed from p
`p.clean()` |replaces "bad" metacharacters with underscores |

Useful Variables
`letters` | abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ 
`digits` | 0123456789 | | punctuation | !"#$%&'()*+,-./:;<=>?@[\]^_{|}~` |

letters is great when used with the line counter "n"...for example, to name things a,b,c,d just use "letters[n]"

Useful flags
`--keep_false` | Normally pyp filters out anything that tests `False` ([],'',0,etc). With this flag, pyp just prints out a blank line if something tests False.