Cached web page server

I'm a big fan of computers from 90s and Internet of that time. Early Linux distros, first complex desktop managers like Gnome and KDE, first media apps like XMMS, even Windows 3.11 and 9x had their charm. So was the early Internet, created with help of nothing more than simple text editors, consisting mostly of pure HTML, full of colorful JPEGs and GIFs. I really miss those times...

While I'm aware of pages like Internet Archive Wayback Machine and totaly awesome The Old Net, it's almost impossible to use these on retro computers running outdated web browsers. Another thing is that connecting PC from 90s to modern Internet not only raises a huge security risk, also accessing any modern webpage would be simply not possible - current revision of HTML language and everything-SSL will prevent you from that. There is another way though.

The approach I've taken requires setting up a FreeBSD server on a Raspberry Pi 3B, which will serve web page dumps as vhosts using Lighttpd HTTP server and Dnsmasq working as a local DNS server, redirecting all DNS queries to Raspberry Pi's HTTP server.

In addition ProFTPD FTP server and Samba SMB server will be set to serve installation files and drivers of your choice and allow you to easilly upload new cached web pages.

In another article I will cover setting up an IMAPS and POPS to POP proxy and SMTP over SSL with authentication to plain SMTP without authentication proxy, allowing old email clients to use modern email services. To even improve the experience, a dial-in server to which you can directly hook up any modem can also be set based on a Cisco router like C1760. I've been able to set it up and will provide a guide on that.

Our retro Internet gateway will allow easy adding and removing web pages by simply placing them in a directory named after the target domain name, like f.e. be.com in a specific location. Two Python apps will take care of updating Lighttpd and Dnsmasq configuration and generating the index page holding a list of all available cached web sites.

In the end I've found out that Apache, while generally more resource hungry, provides better experience than Lighttpd as it can serve pages in a case insensitive manner. In the old days of DOS and Windows almost nobody really cared about using the same case for file name and it's reference in code.

I did my best to make this guide as simple to follow as possible so if you have any experience in Linux or BSD you shouldn't run into much trouble.

Install FreeBSD on Raspberry Pi by flashing the disk image on a microSD card. You can fetch the latest FreeBSD RPi 3 image from here.
Perform initial setup, create a non-root user (type $ useradd --help to get more info). Login with that account and escalate your privileges using $ su or $ sudo -s command, depending on your preference - all actions below have to be performed using privileged account (root).

Perform network and time setup in /etc/rc.conf, in this guide I will call the host revgate:

hostname="revgate.YOUR.DOMAIN"
ifconfig_ue0="inet RPI.LAN.IP netmask NETMASK"
defaultrouter="YOUR.GATEWAY.IP"
ntpd_enable="YES"
ntpd_sync_on_start="YES"

Set date:
```
$ date yymmddhhmm.ss
```
OPTIONAL: Localization
1. Configure locales in /etc/login.conf, example below covers Polish locales:
```
polish|Polish Users Accounts:\
:charset=UTF-8:\
:lang=pl_PL.UTF-8:\
:tc=default:
```
2. Create binary config from the /etc/login.conf file:
```
$ cap_mkdb /etc/login.conf
```
3. Set language for your non-root user using vipw command:
```
username:password_hash:uid:gid:LANGUAGE:0:0:comment:home_dir:shell
```
4. Set keymap in /etc/rc.conf, in this example Polish Programmers using UTF-8 encoding:
```
keymap=pl_PL.UTF-8
```
Perform intial setup of package manager:
```
$ pkg
```
Confirm when prompted.
Update local package repository:
```
$ pkg update
```

Install basic tools:

$ pkg install wget vim-console mc-light sudo tmux

Install current timezone data:

$ cd ~
$ wget ftp://ftp.iana.org/tz/tzdata-latest.tar.gz
$ tar zxvf tzdata-latest.tar.gz
$ mkdir zoneinfo
$ zic -d zoneinfo -p Europe/Warsaw -m 444 -L leapseconds -y yearistype.sh africa antarctica asia australasia etcetera europe factory northamerica southamerica systemv
$ cp zone.tab zoneinfo
$ mv /usr/share/zoneinfo /usr/share/zoneinfo_orig
$ mv zoneinfo /usr/share

Setup timezone:
```
$ tzsetup
```
Update ports tree:
```
$ portsnap fetch && portsnap extract
```
Install ProFTPD FTP server, you can use it to upload new web page caches or serve some files like drivers, programs, etc. to your retro gear, it's not necessary but highly suggested as almost every retro OS supports FTP. If you don't want to use FTP, skip to point 19.:
```
$ pkg install proftpd
```
Enable ProFTPD in /etc/rc.conf by setting proftpd_enable="YES".
Create a scoreboard file:
```
$ touch /var/run/proftpd.scoreboard
```

Setup FTP directories:

$ mkdir /usr/home/ftp
$ cd /usr/home/ftp
$ mkdir upload pub
$ chown nobody:nogroup upload
$ chmod 5777 upload
$ chmod 5775 pub

Configure ProFTPD by editing the /usr/local/etc/proftpd.conf file:

ServerName          "RevGate FTP Server"
ServerType          standalone
DefaultServer           on
ScoreboardFile          /var/run/proftpd/proftpd.scoreboard
SystemLog           /var/log/proftpd.sys
TransferLog         /var/log/proftpd.xfer
ServerLog           /var/log/proftpd.serv
Port                21
UseIPv6             off
Umask               022
MaxInstances            1000
CommandBufferSize       512
User                nobody
Group               nogroup
DefaultRoot         /usr/home/ftp
AllowOverwrite          on
AllowRetrieveRestart        on
UseEncoding         cp1250 cp1250
<Limit SITE_CHMOD
DenyAll
</Limit>
<Limit LOGIN>
DenyAll
AllowUser YOURUSERNAME
</Limit>
<Limit ALL>
DenyAll
</Limit>
<Limit CDUP CWD LIST PWD>
AllowAll
</Limit>
<Directory /usr/home/ftp/upload>
<Limit STOR STOU>
AllowAll
</Limit>
</Directory>
<Directory /usr/home/ftp/pub>
<Limit READ>
AllowAll
</Limit>
</Directory>
<Directory /usr/home/ftp>
<Limit ALL>
AllowUser hekarath
</Limit>
</Directory>
MaxClients 100
<Anonymous /usr/home/ftp>
<Limit LOGIN>
AllowAll
</Limit>
<Directory /usr/home/ftp/upload>
<Limit ALL>
  AllowAll
</Limit>
</Directory>
<Directory /usr/home/ftp/pub>
<Limit READ>
  AllowAll
</Limit>
</Directory>
User ftp
Group ftp
UserAlias anonymous ftp
RequireValidShell off
</Anonymous>

Add FTP user:
```
$ vipw
```
```
ftp:*:14:14:ftp:0:0:FTP User:*:*
```
Start ProFTPD server:
```
$ service proftpd start
```
Install Samba SMB server, it can serve the very same purpose as the previously set FTP server. If you don't need it, simply skip to point 23.:
```
$ pkg install samba48
```

Enter Samba configuration in /usr/local/etc/smb4.conf. pub folder is read-only, you can put there installation files for drivers and programs, etc. upload is read-write to everyone, same goes for cache where you can place your cached web pages:

[global]
dns proxy =  no
load printers = no
disable spoolss = yes
guest account = nobody
map to guest = Bad Password
workgroup = HNET
server string = RevGate SMB Server
netbios name = RevGate
hostname lookups = yes
local master = no
wins support = no
security = user
create mask = 0777
directory mask = 0777
[pub]
path = /usr/home/ftp/pub
printable = no
writeable = no
browseable = yes
only guest = yes
[upload]
path = /usr/home/ftp/upload
printable = no
writable = yes
browseable = yes
only guest = yes
[cache]
path = /usr/home/www/cache
printable = no
writable = yes
browseable = yes
only guest = yes

Enable Samba in /etc/rc.conf:
```
samba_server_enable="YES"
```
Start Samba:
```
$ service samba_server start
```
Install Lighttpd server:
```
$ pkg install lighttpd
```
Enable Lighttpd in /etc/rc.conf:
```
lighttpd_enable="YES"
```
Create WWW directory for the Index file:
```
$ mkdir -p /usr/home/www/data
```

Update main lighttpd configuration in /usr/local/etc/lighttpd/lighttpd.conf:

var.log_root    = "/var/log/lighttpd"
var.server_root = "/usr/local/www/lighttpd"
var.state_dir   = "/var/run"
var.home_dir    = "/var/run/lighttpd"
var.conf_dir    = "/usr/local/etc/lighttpd"
var.vhosts_dir  = server_root + "/vhosts"
var.cache_dir   = "/var/cache/lighttpd"
var.socket_dir  = home_dir + "/sockets"
include "modules.conf"
server.port = 80
server.use-ipv6 = "disable"
server.username  = "www"
server.groupname = "www"
server.document-root = "/usr/home/www" + "/data"
server.pid-file = state_dir + "/lighttpd.pid"
server.errorlog             = log_root + "/error.log"
include "conf.d/access_log.conf"
include "conf.d/debug.conf"
server.event-handler = "freebsd-kqueue"
server.network-backend = "writev"
server.max-fds = 2048
server.stat-cache-engine = "simple"
server.max-connections = 1024
"index.xhtml", "index.html", "index.htm", "default.htm", "index.php"
)
url.access-deny             = ( "~", ".inc" )
$HTTP["url"] =~ "\.pdf$" {
server.range-requests = "disable"
}
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi", ".scgi" )
include "conf.d/mime.conf"
include "conf.d/dirlisting.conf"
server.follow-symlink = "enable"
server.upload-dirs = ( "/var/tmp" )
include "/usr/local/etc/lighttpd/vhosts.d/*.conf"
$SERVER["socket"] == "0.0.0.0:80" { }

Start Lighttpd:
```
$ service lighttpd start
```
Install Dnsmasq server:
```
pkg install Dnsmasq
```

Update main Dnsmasq configuration in /usr/local/etc/dnsmasq.conf:

resolv-file=/etc/resolv.dnsmasq
strict-order
server=/localnet/YOUR.MAIN.DNS.IP
interface=ue0
no-dhcp-interface=ue0
conf-file=/usr/local/etc/dnsmasq.revgate

Sample redirections for /usr/local/etc/dnsmasq.revgate:
```
address=/be.com/RPI.LAN.IP
address=/wp.pl/RPI.LAN.IP
```
Enable Dnsmasq in /etc/rc.conf:
```
dnsmasq_enable="YES"
```
Start Dnsmasq
```
$ service dnsmasq start
```
Place the revcachescan.py in /usr/local/share/revgate directory and link the app to /usr/local/bin.

Add revcachescan.py to crontab:

$ crontab -e

*/15 * * * * /usr/local/bin/revcachescan.py

That's it. Now you should be able to access cached web pages using your favorite retro web browser.