Cached web page server

I'm a big fan of computers from the 90s and the Internet of that time. Early Linux distros, the first complex desktop environments like GNOME and KDE, the first media players like XMMS; even Windows 3.11 and 9x had their charm. So did the early Internet, built with nothing more than simple text editors, consisting mostly of pure HTML and full of colorful JPEGs and GIFs. I really miss those times...

While I'm aware of sites like the Internet Archive Wayback Machine and the totally awesome The Old Net, it's almost impossible to use them on retro computers running outdated web browsers. Moreover, connecting a PC from the 90s to the modern Internet not only poses a huge security risk; accessing any modern web page would simply be impossible, as current HTML and SSL everywhere will stop you. There is another way, though.

The approach I've taken requires setting up a FreeBSD server on a Raspberry Pi 3B, which serves web page dumps as virtual hosts using the Lighttpd HTTP server, with Dnsmasq acting as a local DNS server that redirects queries for the cached domains to the Raspberry Pi's HTTP server.

In addition, the ProFTPD FTP server and the Samba SMB server will be set up to serve installation files and drivers of your choice, and to let you easily upload new cached web pages.

In another article I will cover setting up an IMAPS/POP3S to POP3 proxy and an authenticated SMTP over SSL to plain, unauthenticated SMTP proxy, allowing old email clients to use modern email services. To improve the experience even further, a dial-in server to which you can directly hook up any modem can be set up based on a Cisco router like the C1760. I've managed to set it up and will provide a guide on that as well.

Our retro Internet gateway makes adding and removing web pages easy: simply place them in a directory named after the target domain, e.g. be.com, in a specific location. Two Python apps take care of updating the Lighttpd and Dnsmasq configuration and generating an index page listing all available cached websites.

In the end, I found out that Apache, while generally more resource-hungry, provides a better experience than Lighttpd, as it can serve pages case-insensitively. In the days of DOS and Windows, hardly anyone cared about using the same case for a file name and its reference in code.

I did my best to make this guide as easy to follow as possible, so if you have any experience with Linux or BSD, you shouldn't run into much trouble.

Install FreeBSD on the Raspberry Pi by flashing the disk image onto a microSD card. You can fetch the FreeBSD image for the RPi 3 here.
Perform the initial setup and create a non-root user (type $ adduser and follow the prompts). Log in with that account and escalate your privileges using the $ su command; all actions below have to be performed as a privileged user (root).

Configure networking and time in /etc/rc.conf; in this guide the host is called revgate:

hostname="revgate.YOUR.DOMAIN"
ifconfig_ue0="inet RPI.LAN.IP netmask NETMASK"
defaultrouter="YOUR.GATEWAY.IP"
ntpd_enable="YES"
ntpd_sync_on_start="YES"

Set the date:
```
date yymmddhhmm.ss
```
OPTIONAL: localization.
1. Configure locales in /etc/login.conf; the example below covers Polish:
```
polish|Polish Users Accounts:\
	:charset=UTF-8:\
	:lang=pl_PL.UTF-8:\
	:tc=default:
```
2. Rebuild the login capability database from the /etc/login.conf file:
```
cap_mkdb /etc/login.conf
```
3. Set the login class for your non-root user (polish in the example above) using the vipw command:
```
username:password_hash:uid:gid:LANGUAGE:0:0:comment:home_dir:shell
```
4. Set the keymap in /etc/rc.conf; this example uses Polish Programmers with UTF-8 encoding:
```
keymap=pl_PL.UTF-8
```
Perform the initial setup of the package manager:
```
pkg
```
Confirm when prompted.
Update the local package repository catalogue:
```
pkg update
```

Install basic tools:

pkg install wget vim-console mc-light sudo tmux

Install current time zone data:

cd ~
wget https://data.iana.org/time-zones/tzdata-latest.tar.gz
tar zxvf tzdata-latest.tar.gz
mkdir zoneinfo
zic -d zoneinfo -p Europe/Warsaw -m 444 -L leapseconds -y yearistype.sh africa antarctica asia australasia etcetera europe factory northamerica southamerica systemv
cp zone.tab zoneinfo
mv /usr/share/zoneinfo /usr/share/zoneinfo_orig
mv zoneinfo /usr/share

Set the time zone:
```
tzsetup
```
Update the ports tree:
```
portsnap fetch && portsnap extract
```
Install the ProFTPD FTP server. You can use it to upload new web page caches or to serve files such as drivers and programs to your retro gear. It's not necessary, but highly recommended, as almost every retro OS supports FTP. If you don't want to use FTP, skip to step 19:
```
pkg install proftpd
```
Enable ProFTPD in /etc/rc.conf by setting proftpd_enable="YES".

Create a scoreboard file:

mkdir -p /var/run/proftpd
touch /var/run/proftpd/proftpd.scoreboard

Set up the FTP directories:

mkdir /usr/home/ftp
cd /usr/home/ftp
mkdir upload pub
chown nobody:nogroup upload
chmod 5777 upload
chmod 5775 pub

Configure ProFTPD by editing the /usr/local/etc/proftpd.conf file:

ServerName			"RevGate FTP Server"
ServerType			standalone
DefaultServer			on
ScoreboardFile			/var/run/proftpd/proftpd.scoreboard
SystemLog			/var/log/proftpd.sys
TransferLog			/var/log/proftpd.xfer
ServerLog			/var/log/proftpd.serv
Port				21
UseIPv6				off
Umask				022
MaxInstances			1000
CommandBufferSize		512
User				nobody
Group				nogroup
DefaultRoot			/usr/home/ftp
AllowOverwrite			on
AllowRetrieveRestart		on
UseEncoding			cp1250 cp1250
<Limit SITE_CHMOD>
  DenyAll
</Limit>
<Limit LOGIN>
  DenyAll
  AllowUser YOURUSERNAME
</Limit>
<Limit ALL>
  DenyAll
</Limit>
<Limit CDUP CWD LIST PWD>
  AllowAll
</Limit>
<Directory /usr/home/ftp/upload>
  <Limit STOR STOU>
    AllowAll
  </Limit>
</Directory>
<Directory /usr/home/ftp/pub>
  <Limit READ>
    AllowAll
  </Limit>
</Directory>
<Directory /usr/home/ftp>
  <Limit ALL>
    AllowUser YOURUSERNAME
  </Limit>
</Directory>
MaxClients 100
<Anonymous /usr/home/ftp>
  <Limit LOGIN>
    AllowAll
  </Limit>
  <Directory /usr/home/ftp/upload>
    <Limit ALL>
      AllowAll
    </Limit>
  </Directory>
  <Directory /usr/home/ftp/pub>
    <Limit READ>
      AllowAll
    </Limit>
  </Directory>
  User ftp
  Group ftp
  UserAlias anonymous ftp
  RequireValidShell off
</Anonymous>

Add the FTP user:
```
vipw
```
```
ftp:*:14:14:ftp:0:0:FTP User:*:*
```
Start the ProFTPD server:
```
service proftpd start
```
Install the Samba SMB server. It can serve the very same purpose as the FTP server set up earlier. If you don't need it, simply skip to step 23:
```
pkg install samba48
```

Configure Samba in /usr/local/etc/smb4.conf. The pub share is read-only; you can put installation files for drivers, programs etc. there. upload is writable by everyone, and so is cache, where you can place your cached web pages:

[global]
    dns proxy =  no
    load printers = no
    disable spoolss = yes
    guest account = nobody
    map to guest = Bad Password
    workgroup = HNET
    server string = RevGate SMB Server
    netbios name = RevGate
    hostname lookups = yes
    local master = no
    wins support = no
    security = user
    create mask = 0777
    directory mask = 0777
[pub]
    path = /usr/home/ftp/pub
    printable = no
    writeable = no
    browseable = yes
    only guest = yes
[upload]
    path = /usr/home/ftp/upload
    printable = no
    writable = yes
    browseable = yes
    only guest = yes
[cache]
    path = /usr/home/www/cache
    printable = no
    writable = yes
    browseable = yes
    only guest = yes

Enable Samba in /etc/rc.conf:
```
samba_server_enable="YES"
```
Start Samba:
```
service samba_server start
```
Install the Lighttpd server:
```
pkg install lighttpd
```
Enable Lighttpd in /etc/rc.conf:
```
lighttpd_enable="YES"
```
Create the WWW directory for the index file:
```
mkdir -p /usr/home/www/data
```

Update the main Lighttpd configuration in /usr/local/etc/lighttpd/lighttpd.conf:

var.log_root    = "/var/log/lighttpd"
var.server_root = "/usr/local/www/lighttpd"
var.state_dir   = "/var/run"
var.home_dir    = "/var/run/lighttpd"
var.conf_dir    = "/usr/local/etc/lighttpd"
var.vhosts_dir  = server_root + "/vhosts"
var.cache_dir   = "/var/cache/lighttpd"
var.socket_dir  = home_dir + "/sockets"
include "modules.conf"
server.port = 80
server.use-ipv6 = "disable"
server.username  = "www"
server.groupname = "www"
server.document-root = "/usr/home/www" + "/data"
server.pid-file = state_dir + "/lighttpd.pid"
server.errorlog             = log_root + "/error.log"
include "conf.d/access_log.conf"
include "conf.d/debug.conf"
server.event-handler = "freebsd-kqueue"
server.network-backend = "writev"
server.max-fds = 2048
server.stat-cache-engine = "simple"
server.max-connections = 1024
index-file.names = (
  "index.xhtml", "index.html", "index.htm", "default.htm", "index.php"
)
url.access-deny             = ( "~", ".inc" )
$HTTP["url"] =~ "\.pdf$" {
  server.range-requests = "disable"
}
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi", ".scgi" )
include "conf.d/mime.conf"
include "conf.d/dirlisting.conf"
server.follow-symlink = "enable"
server.upload-dirs = ( "/var/tmp" )
include "/usr/local/etc/lighttpd/vhosts.d/*.conf"
$SERVER["socket"] == "0.0.0.0:80" { }

Start Lighttpd:
```
service lighttpd start
```
Install the Dnsmasq server:
```
pkg install dnsmasq
```

Update the main Dnsmasq configuration in /usr/local/etc/dnsmasq.conf:

resolv-file=/etc/resolv.dnsmasq
strict-order
server=/localnet/YOUR.MAIN.DNS.IP
interface=ue0
no-dhcp-interface=ue0
conf-file=/usr/local/etc/dnsmasq.revgate

Sample redirections for /usr/local/etc/dnsmasq.revgate:
```
address=/be.com/RPI.LAN.IP
address=/wp.pl/RPI.LAN.IP
```
Enable Dnsmasq in /etc/rc.conf:
```
dnsmasq_enable="YES"
```
Start Dnsmasq:
```
service dnsmasq start
```
Place revcachescan.py in the /usr/local/share/revgate directory and link it to /usr/local/bin.

Add revcachescan.py to the crontab:

crontab -e

*/15 * * * * /usr/local/bin/revcachescan.py

That's it. Now you should be able to access cached web pages using your favorite retro web browser.