SPIP 2.0

SPIP and web publishing

SPIP is a system for creating and managing websites. Its editing interface encourages the creation of clear editorial structures which are both long-lasting and flexible [1]. To install SPIP it is necessary to fill in only 5 web forms and it is then ready for use. It makes available the best current technologies for building a website, whether on one’s own or with a team of authors.

For SPIP 2.0 the ergonomy of the editing interface has been much improved. The XHTML and CSS norms are better respected. The pages are much faster due to the systematic use of AJAX [2]. The attaching of documents of different kinds has been unified, while still allowing for a range of usage. This progressive “polishing” means that new users inherit an interface which has been tested, while long-time users will keep their (good) habits.

-  Sets of symbolic URLs

The new, so-called “clean” URLs make it possible to have several URL names pointing to the same object. A particular advantage of this is that when you change the title of an article a new URL is generated without invalidating the old one. In addition, an HTTP client requesting the old URL will be definitively redirected to the new URL.

These URLs are now managed by a separate SQL table of which they are the primary key, which accelerates their use. Compatibility with the previous system of clean URLs is assured by the update mechanism.

These new URLs represent the completion of a project which Togg began to work on before his death (http://toggg.com/spip/spip.php?article19). This new version of SPIP is dedicated to him.

A new set of URLs, called arbo, has also been introduced. It offers a tree notation such as

http://www.mysite.com/sector/section1/section2/article

Finally, it is now possible to choose which URLs to use with a simple click in a table on one of the administration pages. The old method, using the global PHP variable $type_urls but it is advisable to remove this variable from the mes_options.php file in order to have the administration table appear. If you have a customised set of URLs in the urls directory of your SPIP_PATH, then it will automatically be included in the admin table.

-  Inserting media in text

The models img and emb, which allow one to insert a document within a body of text, have been completely rethought. They now follow the naming scheme of the official MIME types, using similarly named filters. This design makes a fine adjustment of any kind of document possible. It is described in more detail in article 3715 which builds upon the article Using models which was written for the release of SPIP 1.9.1.

Authors can thus insert more types of documents than beforehand. It is with this mechanism that SPIP automatically presents the contents of a CSV file (saved from a spreadsheet, for example) as an HTML table. In a similar way, the contents of a txt file is placed within a pre tag, so as to appear verbatim. Finally, in the case of an HTML document its stylesheets are automatically imported. This makes it easy to transform a static site into a SPIP site.

-  Posting documents in forums

A new option allows visitors to post documents in forums. Of course, this option is not activated by default.

The option is controlled by listing the permitted extensions, for example, gif,png,jpg,mp3. The sign * indicates that all document types are accepted, apart from those which have a possible security risk.

-  Many new features for petitions

The many new features are described in Les pétitions sous SPIP: searching for and complex sorting of signatures, paging, RSS feeds, daily and monthly statistics, spreadsheet interface, and multilingual petitions.

-  The management of concurrent editing

The editing area od SPIP manages the concurrent editing by multiple users of articles, sections, news items, keywords, authors and sites.

The following scenario demonstrates the system: Alice and Bob both open the same article at the same time, and each of them try to save their changes, first Alice, then Bob.

• If Alice has changed only the title and Bob only the text, then both changes are accepted. Bob’s title (unchanged) does not squash Alice’s change;
• If Alice and Bob have both made changes to the title:
  • if the titles are identical, there is no problem;
  • if the title is different then Bob receives a notification showing him the title he is proposing to save, the title which is in the database (Alice’s), the difference between the two, together with a form in which he can cut and paste his changes. The title he tried to save is not saved in the database.

Elements are dealt with independently, field by field. So if there is a conflict over the title but not over the standfirst, then the standfirst is saved and the notification only concerns the title.

-  Improvements to plugin installation

— Manual and/or automatic installation

Plugins are installed in a directory /plugins. By creating a sub-directory /plugins/auto, one activates the automatic installation of plugins.

Using the automatic installation system it is possible to add:

• individual plugins, by giving the URL of a zip file,
• a series of plugins listed in an RSS feed.

— Finding plugins

Three sites give users access to plugins according to the end they have in view (developers, documented contributions, themed with compatibility details):

• plugins developed on the Zone can, if their developers wish, be made available in the directory http://files.spip.org/spip-zone/;
• SPIP-Contrib is being developed to make it easier to find plugins;
• the site Plugins.spip, whose launch coincides with that of this version of SPIP, provides a finer referencing of plugins — by theme, by compatibility with different SPIP versions, by language, etc.

The sites Plugins.spip and SPIP-contrib offer RSS feeds which are usable straightaway for the installation of plugins compatible with SPIP 2.0

-  Connecting to multiple databases and servers, partial fusion, and automatic templates

SPIP can now be used not only with MySQL but also with PostgreSQL and with the SQLite 2 and 3 extensions of PHP. The choice is given during the installation process and then kept in the file holding the connection information (/config/connect.php). Note: the format of the backup file is the same for all versions. This means it is possible to install a site saved from one SQL server on a different kind of server.

This new portability of SPIP means that templates can been used on several different SQL servers at once — a feature which has been lacking up to now.

Through the Site maintenance menu you can access a form similar to the one used during installation and which allows you to enter the connection codes for another database. This produces a connection file similar to that for the principal site and which is sufficient to make it possible to use the database. If the database is also a SPIP site, it is sufficient simply to copy that site’s connection file.

Once this connection information has been entered you can then use your templates for the other database, simply by adding an additional parameter, connect, to the URL. The value of the parameter should be the name of the connection file (without the .php extension). It is also possible to mix, in one template, loops using various databases: SPIP opens the necessary connections and passes from one to another as needed. It is possible for these databases to be managed by other SQL servers than the one used by the principal site, and in this case SPIP acts as a kind of aggregator of extracts from several databases.

To make it possible to have a quick overview of the tables of a database, an automatic template is created when the value of the page parameter of the URL is in this form: table:tablename.

This skeleton function shows the table contents in different navigation modes and allows you to save the template which is created.

Thus it is easy to use SPIP as a utility for managing database content, even content implicating multiple servers.

For reasons of confidentiality this mechanism is only accessible to the main webmaster. However, it can be opened more widely by customising the function autoriser_webmestre.

Another mechanism which has been stabilised is making a backup of only a part of the database, and the possibility of merging such backups into another site.

You can find a complete presentation of the functionalities concerning SQL databases in this article: Les bases de données en SPIP. Of particular note is a naming system for connection files which allows all the databases of installed sites to be aware of each other without need for any additional declaration.

-  Using one copy of SPIP for multiple sites

Sharing one copy of SPIP for several sites has been possible since SPIP 1.9. The system can now be used whatever directories are used for SPIP data: the limitation previously imposed for the IMG directory has been removed.

The system is recommended for webmasters who have several sites because it can make maintenance a lot easier. It also offers a greater ease of use if they wish to use multiple databases, as described in this article.

We encourage web hosting providers to investigate this functionality which allows them to economise on disk space and on downloads of multiple copies of SPIP. Several extensions which set up configurations automatically have been developed. For these see http://www.spip-contrib.net/Mutuali.... In a configuration like this, SPIP works as a kind of blog farm or a CMS server.

SPIP and web design

-  Templates based on CSS from LayoutGala

SPIP comes with modern, structured standard templates ready for immediate use. It also has a language (loops, tags, filters) which webmasters can use to customise their site and go beyond the standard templates.

The standard templates ("dist") are based on the HTML structure from Layout Gala. These allow you to move elements simply by changing a few lines of CSS:

<div id="page">
	<div id="entete">Header</div>
	<div id="conteneur">
		<div id="contenu">Contents</div>
	</div>
	<div id="navigation">Navigation</div>
	<div id="extra">Extra</div>
	<div id="pied">Footer</div>
</div>

The default HTML code of the templates includes Microformats.

The templates are XHTML 1.0 strict compliant (as are the pages of the editing interface). However, the declared Doctype is XHTML transitional, so as not to oblige authors to learn a difficult and debatable standard (see the first paragraph of The integrated XML validator).

There are new ways to write shortcuts and new variables allow for greater customisation (Cf. : http://archives.rezo.net/spip-core....).

-  Customising the tags which SPIP inserts for italics and bold-face:

$debut_italique = '<i>' ;
$fin_italique = '</i>'  ;
$debut_gras = '<strong>' ;
$fin_gras = </strong>' ;

-  Paragraphs: all texts are now paragraphed by SPIP in a coherent, identical way (even texts comprising only one paragraph). In order to override this and force the previous behaviour it suffices to set the variable $toujours_paragrapher to false, thus:

$GLOBALS['toujours_paragrapher'] = false;

-  Removal of the class="spip" on p, i, strong and li tags. The customisation variable for reinstating them is (note the leading space!):

$class_spip = ' class="spip"';

N.B. If <p class="spip"> is reinstated, then in footnotes <p class="spip_note"> is also retained for the sake of compatibility, otherwise it also disappears.

More finely graded intervention is also possible. For example, in order to remove class="spip" from paragraphs, but leave them for italics:

$class_spip = '';
$debut_italique = '<i class="spip">';
$debut_gras = '<strong class="spip">';

In fact, if you really want to remove class="spip" from all the other tags (ul, ol, tables, hr, h3 and blockquote generated by SPIP shortcuts), you can do this:

$class_spip_plus = '';

-  Poetry: the code generated by the SPIP shortcut <poesie> is now: <blockquote class="spip_poesie">

-  Date of first publishing: let us note here another customisation variable, although it does not concern CSS styles. By default, the year of the publication date can be freely chosen. To make the list begin from a certain year the variable $debut_date_publication can be customised thus:

$debut_date_publication = 1997;

-  Improved and new filters

— image_typo works (in experimental mode) for Arabic (and probably Hebrew too). The presence of Arabic and Hebrew characters is automatically detected setting off a process which both reverses the order of letters and manages the complex ligatures of Arabic.

— Three new filters for manipulating colours are now available: couleur_saturation, couleur_web et couleur_4096.

The filter couleur_saturation takes a value between 0 and 1 to indicate the colour saturation:

• 0: white
• 1: completely saturated colour (N.B. A completely saturated colour is not black).

This filter is useful for making a monochrome image for cases in which couleur_eclaircir and couleur_foncer are not sufficiently precise and, above all, in which the colours fade (because only the brightness has been adjusted and not the saturation).

The filter couleur_web gives a colour approximation using one of 256 colours. The aim is not really to achieve a display for screens with 256 colours (do such screens still exist?), but more to limit the number of colours which can be applied to a graphical filter (for example, colouring a typographical image, or changing a stylesheet) and thus avoiding the possible production of 16 million variants of a file.

The filter couleur_4096 gives a colour approximation using one of 4096 colours.

— Finally, the filter couleur_extreme now takes an optional parameter with a value between 0 and 1 (0.5 by default) in order to regulate the threshold at which the colour tips into black or white.

The functions which convert between RGB, HSL et HSV colours are now included with the core SPIP code (they are used internally by many of the filters which manipulate colours). HSL is now used by couleur_extreme, which means that the black/white tipping is decided on a “visual” basis, and not only on an average of the components.

— The graphics filter image_aplatir now accepts a 4th parameter which makes it possible to preserve transparency (particularly useful for conversions to GIF format).

— The new filter image_format allows you to force an image from one format to another, while preserving transparency in the case of PNG and GIF formats. In other words this filter is a simple shortcut for the image_aplatir function.

N.B. When a PNG image is converted, with transparency, to GIF, this function does not allow you to choose which colour you want for the conversion of semi-transparent points. If this is a problem, then image_aplatir should be used.

— The image filters now have a mechanism for cleaning up temporary images. So after the application of a series of several filters in one expression, only the final image is retained. This cuts down on the disk space occupied.

— The new filter compacte_head compresses and combines the Javascript and CSS files called by a template into just two files. This helps page loading speed (instead of several Javascript and CSS files being called only one of each is inserted, and these are smaller in size). If you use an inclusion inc-head.html (as does the dist template) which contains all the Javascript and CSS calls, then this function can be automatically activated if the option to do so has been set in SPIP’s configuration page.

-  New inclusion modes, with AJAX in particular

The tag #INCLURE interprets the argument {env} as transmitting the whole environment to the included template. This notation also works with <INCLURE ...>. If other arguments are also present then their values take precedence over the values of the environment:

 <INCLURE{fond=notebook/section}{env}{lang=en}>

will include notebook/section and pass the {id_rubrique} et {debut_pages} from the environment (from the URL, that is to say), but forcing the language to English, even if the URL contains ?lang=XX.

The argument {ajax}, when it is added to the tag #INCLURE or <INCLURE ...> makes it possible to “ajax” automatically all the links which carry a CSS Ajax class. It has the same effect on all links contained in the class “pagination”. For more information, see `ajax` for inclusions.

Note that the INCLURE tag, in order to come closer to XML syntax, can take a final slash.

Another example of inclusion are the so-called dynamic tags whose names begin with #FORMULAIRE_ and which produce web forms. The forms, used on the public site and also in a part of the editing interface, have been rewritten using a new method, CVT (charge/load, verify, treat). This method makes it possible to create new forms simply by writing templates. SPIP looks after checking for badly filled-out forms and writing correct data to the database.

The directory prive/contenu/ contains templates for rendering objects in the editing area. And prive/editer/ contains templates for the editing of the objects. They call a series of #FORMULAIRE_EDITER_... located in the prive/formulaires directory. These pages can be customised with section numbers, by adding -xx.html in the same way as other templates.

These new possibilities are explained in detail in Formulaires CVT par l’exemple and The CVT forms of SPIP.

-  New and changed tags

— Tags with names which begin with URL_ are processed generically. If the tag has no argument, as is the case with URL_type, then the SPIP compiler calls the function generer_url_entite and applies it to $id_type (which must be defined in the context), and to type. The tags URL_ARTICLE, URL_RUBRIQUE, etc. which used to use the now obsolete functions generer_url_article, generer_url_rubrique etc., work in the same way. If the tag has an argument, then it is to this and not to $id_type that generer_url_entite is applied. This whole group of tags is compatible with connection to multiple SPIP databases (see above). One notable use of this feature is to allow you to test a set of templates on a production site without changing its installation.

— The #EXPOSE tag (see Highlighting an item in a list) now has extended functionality. It takes the primary key of the closest containing loop (or else of the loop designated by the explicit syntax #nom:EXPOSE) and for each iteration it compares the current value with the parameter of the same name in the calling URL. If the two values are equal, it renders its first argument, and otherwise its second. The comparison extends to the hierarchy of the key in question. In other words, in a RUBRIQUES loop the primary key is compared with the section number of the articles under consideration; in a GROUPES_MOTS loop the primary key is compared with the number of the keyword being examined.

— A new tag, #FILTRE, makes it possible to apply a filter to the whole of the rendered page. By placing

#FILTRE{compacte_head}

in a template all the rendered code of the page will be fed through the filter compacte_head.

— It is now possible to customise the follow-on characters of the #INTRODUCTION tag by defining them in mes_fonctions.php (or mes_options.php):

define('_INTRODUCTION_SUITE', ' (...)');

or

define('_INTRODUCTION_SUITE', '');

— The filter couper, which is used by #INTRODUCTION, can also be used independently. It can take an additional parameter indicating which characters to insert at the end of a truncated text:

[(#TITRE|couper{10,'...'})]

— #SESSION is a new tag which gives you access to the information concerning a logged-in user (id_auteur, etc.), and to have separate caches according to the user.

-  New features in criteria

It is now possible to cause a join between two SQL tables by using a notation TABLE.NAME in a criterion. The join is possible if the table given has a field with the same name as the main table of the loop; the primary key takes priority. In the case of criteria without an explicit {TABLE.NAME} operator, the implicit value is the NAME in the current context (in particular at the top level, in the URL). For example here is a loop which looks for signatures of petitions attached to all the translations of one article:

<BOUCLE_signatures(SIGNATURES){articles.id_trad}  />

As the SQL table signatures has, just as the table articles does, a field id_article, giving the value of the field id_trad in the URL causes a selection to be made in the table signatures of all those attached to an article having that value of id_trad

This notation can also be used with conditional criteria For example, here is a loop which list the signatures of either a petition of a given article (when id_article is present in the context), or of the petitions attached to the translations of a same article (if it is id_trad which is present):

<BOUCLE_signatures(SIGNATURES){id_article?} {articles.id_trad ?}  />

This new functionality renders some criteria obsolete (such as titre_mot, for example), but they remain operational for the sake of compatibility.

Another important change for criteria concerns the search engine. Before this version, SPIP used an supplementary SQL table in order to index elements when they were created. SQL servers are now have sufficiently fast search operators to allow this expensive indexing to be dispensed with and the recherche criterion now uses these native SQL search operators. {recherche} now accepts an optional argument giving the string being sought. If it is left out, it is always the URL variable recherche which is used. In other words, <BOUCLE_t(ARTICLES){recherche}> and <BOUCLE_t(ARTICLES){recherche #ENV{recherche}}> are equivalent.

This additional argument can be used in several ways:

• to look for a text given by the internal context in a template. For instance:

  <BOUCLE_m(MOTS){id_mot}>
   <BOUCLE_r(ARTICLES){recherche #TITRE}>...
    </BOUCLE_r>
  </BOUCLE_m>
  

• to communicate the text to look for to an included template:

  <INCLURE{fond=x}{recherche=NNN}>
  

• to use in a model in order to select objects whose fields contain the string:

  <modele1|recherche=spip>
  

If you use the argument combined with a conditional criterion, you must avoid any spaces between the question mark and the argument:

{recherche ?#ENV{rech}}

is good, but

{recherche ? #ENV{rech}}

is not (unless you want to specify a leading space as part of your search string).

-  With Salvatore, my plugin spricht разных språk

Salvatore (the name of a character from The Name of the Rose) is ugly, uncouth, deformed, rather odd, and speaks every language.

Salvatore is our new tool which allows you to develop a multilingual plugin by automatically synchronising language files from SPIP-Zone with the translation area of SPIP (where the translators of spip-trad work). You can consult the documentation here.

-  XML validation

The integrated XML validator has a new feature. If the argument passed is a directory and not a URL, then the validator checks all the files of the directory whose extension matches the ext parameter of the URL. If no such parameter is found, then the extension .php is assumed. And if there is no .php file, then the validator checks .html files instead (they are taken to be templates). The arguments expected by these templates are generated automatically after an analysis of their contents. No such analysis is possible for scripts, PHP being a dynamically typed language. However the general architecture of SPIP, in particular the absence of the exit command, means that even a blind test with the validator can be very informative.

XML’s influence can also be seen in a small innovation of syntax. Loops with no body can now be written in a shortened form:

For example, the following loop:

<BOUCLE_message(FORUMS){id_article}>
</BOUCLE_message>
#TOTAL_BOUCLE messages
</B_message>

can be written in this way:

<BOUCLE_message(FORUMS){id_article} />
#TOTAL_BOUCLE messages
<//B_message>

SPIP and web development

This section describes the development tools used by the SPIP team. Other developers need to master them in order to write extensions.

Many websites have become online applications through the interactivity which they offer to users. A lot of the developments in SPIP 2 are with a view to making it possible to create Web 2.0 sites: managing users, cooperation on the public site, better management of dated events, etc.

-  A virtual SQL server

This version of SPIP required an entire rewriting of the SQL queries in order to make it portable. The porting relies on a programming interface and it is sufficient for a SPIP extension to use the same interface in for it to become portable, in its turn. The interface also manages security vulnerabilities of the code injection type.

This interface does not rely on any pre-existing libraries, because the ones which exist do not have a sufficient level of abstraction to meet all the situations encountered. Instead, it contains about 30 virtual functions deducted from SPIP’s previous code. Their level of generalisation makes it possible to use the interface in order to integrate diverse SQL databases entirely independently of SPIP’s other features.

The interface includes a management of its own versions. This makes it possible to use simultaneously SPIP extensions which require different versions of the interface — in the event of future evolutions.

The rewriting has also advantaged MySQL users, because the most “expensive” SQL constructions have been removed. The coherency of the database is checked incrementally and no longer by a periodic complete check. The global locking of the server to protect write operations has been systematically replaced by SQL constraints which automatically refuse concurrent write operations on the same item.

Some indices have been improved by being redefined. The compiler analyses better which joins are needed, and the automatic SQL produced is, on this point, very often close to the best possible.

One particular way of observing these improvements is by the use of the SQL profiler which is now available in the SPIP debugger.

The upgrade system has also been completely rethought and rewritten. It can now handle large scale databases for which one previously needed to use independent SQL tools.

This aspect of SPIP is described in its own article: L’interface de SPIP avec SQL.

-  New pipelines, used much more widely

There are now almost 50 pipelines with more than 180 entry points.

In particular

• the pipelines pre_edition and post_edition are called systematically before and after writing an edited object to the database, whether at its creation, modification or at a change of status;
• the pipeline recuperer_fond allows you to act upon the result of any included template at the moment of its calculation;
• the pipelines pre_boucle and post_boucle allow you to change the behaviour of a loop without needing to redefine the code.

The greater use of pipelines means that plugins can intervene more deeply in SPIP’s operation with less risk of incompatibilities arising between them.

-  External libraries

SPIP connects to MySQL in the same way as phpMyAdmin: if the extension has not been loaded, then it loads dl('mysql.so') dynamically, having first carefully checked that it is authorised to do so. This makes installation possible on servers which are configured in this way (Ubuntu, for example).

The SPIP function charger_php_extension('mysql') allows you to request a dynamic loading.

Over to you!

The evolution of SPIP now makes it possible to develop not only ordinary websites, but intranet applications and all kinds of online applications, using user sessions, AJAX, customised pages, and advanced interactive interfaces.