Utilities

Testing pattern mining in synthetic noisy tensors

This archive contains the source codes of four useful commands to test pattern mining in synthetic noisy tensors. The first one, gennsets, generates randomly positioned n-sets of any sizes in a null tensor of any sizes. Then, the second one, noise, or the third one, num-noise, turns this "perfect" tensor noisy. The difference between the two commands is that noise outputs another Boolean tensor (the noise only switches Boolean values), wheras num-noise turns the tensor fuzzy (the noise turns Boolean values into degrees of membership). Finally, the fourth command, n-set-diff, provides a tuple-based comparison of the hidden patterns (as output by gennsets) with those discovered in the noisy version of the tensor.

Please, take the time to read the INSTALL files before compiling these commands and the README files before using them.

If you want to reproduce experiments reported in the article introducing Fenster, you may be interested in these Bash scripts. The source code of DCE is not available on the website of its author anymore. Since it was distributed under the terms of the GNU GPL, we are free to redistribute it. Thus, here is the source code of DCE. The only improvement w.r.t. the original code is the added possibility to specify different minimal size constraints for the different dimensions of the tensor.

Beamerposter themes for the DCC

This archive contains a theme beamerposter. The background and most of the colors come from DCC's website. The archive contains as well an example source file, a Makefile, and the resulting PDF. The example poster presents some useful commands to structure it.

To use the theme, beamer and beamerposter are (obviously) required. On Debian and derivatives (including the *buntu), use the package manager to install latex-beamer and texlive-latex-extra. If you do not want the whole texlive-latex-extra package (or if your distribution of choice does not ship beamerposter), visit beamerposter's website.

If you feel nostalgic about DCC's previous website, here is another theme beamerposter that reuses its background.

srtfold

Given in argument a maximal number of characters per line, srtfold reformat .srt subtitles to satisfy that constraint (except for words that, alone, exceed it) and to display at most two lines at a time. Consider for instance those subtitles:

Processing them, srtfold 40 outputs:

And srtfold 20 outputs:

srtfold tries to avoid widows and orphans. It heuristically balances the number of characters per line. It prefers to break lines after punctuation marks. With a lower priority, it prefers longer second lines too. If an input subtitle cannot be displayed within two lines during the provided time interval, that interval is divided into sub-intervals whose duration is proportional to the number of characters. srtfold can process slightly-malformed subtitles (more than one blank lines between subtitles, missing/supernumerary spaces around "-->", missing hours, dots separating seconds from milliseconds, incorrect padding) that it fixes. It removes supernumerary spaces between words, but keeps the input line breaks. It removes HTML tags too. Here is the Shell script. Executing it with no argument makes it print a usage help.

To improve the readability of the output subtitles, some spaces in the input subtitles can be turned non-breaking. Appropriate substitutions depend on the language. Two sed substitutions are commented in the script: to turn non-breaking every space right before ":", "?" or "!" (useful in French) or right after a word consisting of a single alphanumerical character.

seq-phragmen

seq-phragmen is this Shell script (or that one in Portuguese). It implements Phragmén’s ordered method ("formulation 3" on page 22 of Phragmén’s and Thiele’s election methods, by Svante Janson). The method aggregates ballots, which are rankings of candidates in order of preference. It has been used in Sweden since 1921 and satisfies a combination of mathematical properties that is unique (in 2024), in particular the "committee monotonicity" and the "proportional justified representation". Brill et al. prove the latter property in Phragmén’s voting methods and justified representation.

Every candidate must correspond to a line in a file. For instance:

To break a tie, seq-phragmen prefers the candidate before in the file. That file must be passed in argument. If it is the only argument, seq-phragmen interactively runs. It repeatedly lists the candidates, each preceded by a number, and asks for a ballot:

That ballot must be a sequence of numbers in descending order of preference of the related candidates. Except digits and line breaks, any characters can separate the numbers. Those of reproved candidates must not be input. If a number corresponds to no candidate or occurs several time, the ballot is ignored. Those same sequences can be lines in (a) file(s) passed as (an) additional argument(s). If so, seq-phragmen non-interactively runs and empty ballots are ignored, whereas a digit-free sequence in the interactive mode terminates the input.

For example, to non-interactively aggregate 1034 ballots "Alice A. > Bob B. > Carol C.", 519 ballots "Peggy P. > Quinn Q. > Rupert R.", 90 ballots "Alice A. > Bob B. > Quinn Q.", and 47 ballots "Alice A. > Peggy P. > Quinn Q.":

seq-phragmen takes seconds to process the 1690 ballots. It ranks every candidate approved at least once. Writing --verbose (or -v) before the candidate file details the computation:

Those explanations match those on pages 23 and 24 of Phragmén’s and Thiele’s election methods, where the same example is treated. Called with no argument, seq-phragmen shows a usage help.

pdf-page-grep

pdf-page-grep searches patterns (by default, basic regular expressions) in PDF files and concatenate the pages with matches. The arguments starting with "-" (e.g., -F or --ignore-case) are considered options. They are passed to grep. The output file is named after the "basename" of the last matching PDF file followed by "-matches.pdf".

The script is very short and simple. It is an occasion to learn. For this purpose, it is quite heavily commented. Moreover, Lucas Westermann made me the honor to write a pedagogical article about it. This article was published in the issue 89 (pages 10–11) of the Full Circle Magazine, which is freely readable.

If you do not want a PDF output but a text one with the sole matching lines, then install pdfgrep, a fully-featured program that probably is in the repositories of your GNU/Linux distribution.

Installation

Besides grep, the script mainly relies on pdfinfo, pdftotext, pdfunite, and pdfjam. In Debian and derivatives, the package named "poppler-utils" provides the three first commands and "texlive-extra-utils" provides pdfjam. Both packages must be installed. pdf-page-grep was tested on bash and dash but probably works as well on other shells. Just download it and execute it! Running it with no argument makes it print a usage help.

Non-interactive usage

pdf-page-grep can be non-interactively used. To do so, redirect the standard input to a file with one pattern per line and an empty line at the end. pdf-page-grep's exit status is 0 if pages matched the patterns, and 1 otherwise.

offline-emerge

This Bash script allows to keep the power of Gentoo's package manager without a local Internet connection. Every action that requires fetching some files from the Net lists instead, on a removable device (typically a USB key), the URLs of these files. A script on the removable device takes care of the download from any *NIX system (GNU/Linux, BSD, Mac OS X, etc.) connected to the Internet. Back on the Gentoo, a command allows to execute every postponed action. In this way, despite the lack of an Internet connection, it is easy to update the Portage tree, to install a new software or to update the whole system.

Installation

An ebuild is available here. Place it in your overlay and execute:

If you do not have an Internet connection, you first have to manually download the source code and place it among your distfiles.

Finally you have to define the variable MOVINGDIR in /etc/make.conf. This variable must contain the path to the directory offline-emerge will use on your removable device. For example, to use /media/usbdisk/moving-portage as a directory on your removable device:

At the first execution of offline-emerge, the files on the removable device will be created.

Usage

If you can read French, the French version of this section may be useful to you. To know everything about offline-emerge, please read its manual:

trivialibre

According to its Web site, "Trivialibre is a set of questions dealing with Free Software for the famous Trivial Pursuit game". My tiny contribution consists in a small Shell script that asks those questions in graphical dialog boxes. Because the original questions are in French, you probably are not interested in them (if you are, download this archive instead). However you may still be interested in the script as a way to, for instance, study for exams. Indeed, adding/removing questions (resp. categories of questions), is only a matter of editing the files in the "categories" directory (resp. adding/removing files in this directory). The script itself needs not be touched and was translated into English. A few questions from Trivialibre were translated as well to make examples.

Features

• The player chooses a category or select "Random!" for a random selection;
• A 20s delay (with a progression bar) is let for the player to find an answer (or not);
• This delay, in seconds, actually is a default value, hence can be modified (it is the only, facultative, option of the script);
• The yet unasked questions are stored on the disk so that the game can be quit without risking being asked the same questions next time;
• When a category is exhausted, all its questions are taken in a different order.

The script is very short and simple. It is an occasion to learn. For this purpose, it is quite heavily commented. Moreover, Lucas Westermann made me the honor to write a pedagogical article about it. This article was published in the issue 58 (pages 5–7) of the Full Circle Magazine, which is freely readable.

Installation

The script relies on Zenity, which is, I believe, installed on most GNU/Linux distributions (especially those using GNOME). If it is not the case, it probably is in the repositories. The other executed commands should not raise any trouble if your system has a reasonably recent version of GNU coreutils (usage of shuf). The script has been tested on bash, dash and zsh. It should probably work as well on other shells such as ksh.

To install the script, this archive (or this one for hundreds of questions in French) must first be downloaded and uncompressed wherever you want. Then, it is only about executing trivialibre to play. It is, of course, possible to create a launcher (typically in the "Games" menu) and an icon is available in the archive.