Find a way to reproduce sub page listing without relying on static site generator

[renoirb]

We have about 6 000 documents in our main content pages. Regenerating static HTML version, send to a Lucene index every updated document at every merge to master would be intensive for no real benefit.

How about we remove the need for the static site generator to require a full generation run at every change. One reason most static site generator has to require a full run are cases when you want to list folders with sub pages.

Why not leverage an already baked-in feature from the web server and let it generate sub pages for us.

That way our static-site generator won’t need to have this task to be done.

Expected tasks

• Have a proof of concept that fills requirements
• Create web server package with "Directory listing" that we can theme
• Create files in static site generator content source "directory listing templates" to be used by web server
• Commit NGINX configuration to make use of the templates into https://github.com/webplatform/docs along with redirect maps config of issue Ensure original URLs are redirected (301 Moved Permanently) to normalized names, including pages that has redirects #6

[renoirb]

Have a proof of concept that fills requirements 👍

[renoirb]

How to configure NGINX (very rough) notes.

Steps

1. Get source and build dependencies

   wget http://nginx.org/download/nginx-1.9.2.tar.gz
   tar xfz nginx-1.9.2.tar.gz
   apt-get -y install build-essential zlib1g-dev libpcre3 libpcre3-dev libbz2-dev libssl-dev tar unzip
   

2. Get modules we want to use

   git clone https://github.com/aperezdc/ngx-fancyindex.git nginx-fancyindex
   git clone https://github.com/giom/nginx_accept_language_module.git
   git clone https://github.com/simpl/ngx_auto_lib.git
   git clone https://github.com/anomalizer/ngx_aws_auth.git
   git clone https://github.com/openresty/headers-more-nginx-module.git
   

3. Configure and build

   cd nginx-1.9.2/
   ./configure --prefix=/opt/nginx \
           --add-module=../nginx-fancyindex \
           --add-module=../nginx_accept_language_module \
           --add-module=../ngx_auto_lib \
           --add-module=../ngx_aws_auth \
           --add-module=../headers-more-nginx-module
   make
   

   As root

     sudo -s
     make install
   

4. Create Upstart config file

   vi /etc/init/nginx.conf
   

   Add contents

      // file: /etc/init/nginx.conf
      description "nginx - small, powerful, scalable web/proxy server"
   
      start on filesystem and static-network-up
      stop on runlevel [016]
   
      expect fork
      respawn
   
      pre-start script
          [ -x /opt/nginx/sbin/nginx ] || { stop; exit 0; }
          /opt/nginx/sbin/nginx -q -t -g 'daemon on; master_process on;' || { stop; exit 0; }
      end script
   
      exec /opt/nginx/sbin/nginx -g 'daemon on; master_process on;'
   
      pre-stop exec /opt/nginx/sbin/nginx -s quit
   

5. To test FancyIndex, add to a location block

   location / {
       // ...
       fancyindex on;
       // where /foo/directorylisting is available from NGINX itself.
       fancyindex_css_href "/foo/directorylisting/fi.css";
       fancyindex_footer   "/foo/directorylisting/fi_footer.html";
       fancyindex_header   "/foo/directorylisting/fi_header.html";
   }
   

Reference

• NGINX Source
• Build NGINX on Debian Squeeze HOWTO
• NGINX 3rd party modules