LEAFNODE

Overview
Installation
Update from previous versions
1. Update from versions < 2.0
2. Update from versions < 1.9.3
3. Update from versions < 1.6
Files and directories
1. The main configuration file
2. The filter file
Local newsgroups
Deinstallation
Frequently asked questions
Copyright

1. OVERVIEW

Leafnode is a USENET software package designed for small sites, with a few tens of readers and only a slow link to the net. It is developed on Linux, but has been reported to compile and run also under any BSD flavour, Solaris and Irix.

The Leafnode package consists of several programs, three essential ones and several add-ons.

Leafnode is the NNTP server. It talks to the normal news clients, and stores readership data.
Fetchnews is the NNTP news-gatherer. It looks at the readership data Leafnode stores, and selects what groups to pull news from. It also transfers your contributions to one or more upstream servers.
Texpire is responsible for deleting old and uninteresting news. It deletes all discussion threads that are old and not recently read.

These are the add-ons:

Applyfilter lets you delete articles from your local news spool which fit a certain pattern.
Checkgroups inserts the titles of newsgroups into the newsgroup database.
Newsq shows which news are waiting to be transferred to your upstream server(s).
Rnews sorts an UUCP batch or articles downloaded by other means into the news spool.

Only groups that someone has been reading in the past week are fetched from the upstream NNTP server. When someone stops reading a group, fetchnews will stop reading that group a week later (this is the default which can be configured), and when someone starts reading a group, fetchnews will grab all the articles it can in that group the next time it runs.

Leafnode's distinguishing features are:

Leafnode is written with the novice user in mind. It is very easy to install and configure and requires no manual intervention once installed. Leafnode tries very hard to recover automatically from error situations.
Compared to other servers, leafnode uses very little disk space and bandwidth. Of course this also depends on how many users are accessing your server and is probably not true for 300-user sites.
The newsspool can be easily manipulated by small scripts because it is maintained in plain file format. Some tiny examples are in the subdirectory tools/ .

Of course, Leafnode also has weaknesses. Some of these are:

In just any error situation, leafnode tries to fix its problems by deleting the offending article. This can cause leafnode to lose news.
Due to its open architecture, Leafnode scales very badly.

The current version of leafnode is available from http://www.leafnode.org/ .

There is also a leafnode mailing list. Send mail to

leafnode-list@wpxx02.toxi.uni-wuerzburg.de

with "Subscribe" in the Subject: to subscribe.

2. INSTALLATION

Leafnode uses GNU autoconf to determine the configuration of the machine it will be compiled on. Type

         sh ./configure

to coreate an appropriate Makefile and config.h .

The configure script can take some flags:

--with-ipv6       use this if you have and want to use IPV6
--prefix=[dir]    Leafnode installs itself normally in the /usr/local/sbin
                  directory. If you want to use another directory, use this
		  flag. For example, if you want to install leafnode in
		  /usr/sbin, use --prefix=/usr/sbin .

Type

          make

There should be no errors. Dependent on your compiler, you might get the warning:

    /fetchnews.c: In function `nntpactive':
    /fetchnews.c:1062: warning: `%y' yields only last 2 digits of year

Don't worry about it.

Create a user "news" if you don't have one.
Type
```
          make install
```
If you prefer an installation with the binaries stripped free of symbols, use
```
	  make install-strip
```
instead.
If you are updating leafnode from a previous version, see the section "Updating" below.
Edit $(LIBDIR)/config. $(LIBDIR) defaults to /etc/leafnode, but can be adjusted in the Makefile. For documentation, see below or config.example and leafnode(8). It is absolutely necessary to change the "server" parameter to point it to your upstream news server (generally the one of your IP provider). Make sure the environment variable $NNTPSERVER or /etc/nntpserver points to your own host so clients will talk to leafnode rather than try to go to the upstream server.
If you want to use filtering of the incoming spool, see the section on the filter file below.
Set up a cron job (as user "root" or "news") to run texpire every night or maybe every week. Here is my crontab line, which runs nightly:
```
0 4 * * * /usr/local/sbin/texpire
```
I did "crontab -u news -e" as root to edit the crontab file, and added this line. Substituting "1" for the third "*", thus:
```
0 4 * * 1 /usr/local/sbin/texpire
```
tells cron to run texpire at 4am every Monday morning.
Make sure fetchnews is run at the appropriate time. If you have a full-time link, run it out of cron (as "news" again); if not, run it when your connection to the net is established. If it is run as root it will change to user "news". If you use PPP, you can run fetchnews from /etc/ppp/ip-up.
Edit /etc/inetd.conf so that $(BINDIR)/leafnode is executed for incoming NNTP connections. Here is my inetd.conf line:
```
nntp    stream  tcp     nowait  news    /usr/sbin/tcpd /usr/local/sbin/leafnode
```
This starts leafnode for all connections on the nntp port, subject to /etc/hosts.allow and /etc/hosts.deny screening. If you don't have tcp wrappers (i.e. /usr/sbin/tcpd) you might use this line instead:
```
nntp    stream  tcp     nowait  news    /usr/local/sbin/leafnode
```
However, using leafnode without tcp wrappers is strongly discouraged since it opens up your news server to abuse.
After these changes, force inetd to read the changed configuration file by sending it the HANGUP signal. To achieve this, issue the following command (as root):
```
kill -HUP `cat /var/run/inetd.pid`
```
If you have tcpd installed, write the following into /etc/hosts.deny:
```
leafnode: ALL
```
and into /etc/hosts.allow:
```
leafnode: 127.0.0.1
```
to protect your news server from abuse. If you want to make leafnode accessible to additional IP numbers/domains, add them in /etc/hosts.allow in the format described above. See hosts_access(5) and hosts_options(5) (if applicable) for more information about tcp wrappers.
Run fetchnews. The first run will take some time since fetchnews reads a list of all newsgroups from your upstream server. In the worst case, this can take up to 60 minutes, depending on how many newsgroups your provider offers and how reliable your modem connection is. To see fetchnews working, run it with -vvv.
Read news using an NNTP client (with $NNTPSERVER or /etc/nntpserver pointing to your own host). Select the groups you want to read in the future. You will find them empty except a default article. Reading this article is necessary with some newsreaders to select the groups for further fetching.
After this, you should have empty files in /var/spool/news/interesting.groups/ for every group you want to read.
Run fetchnews again. This run should pick up all the groups you want to read.

3. UPDATE FROM PREVIOUS VERSIONS

3.1 Update from versions < 2.0

From 1.9.x to 2.0 there were some changes to options in the main configuration file. The options "maxage", "maxlines", "minlines", "maxbytes" and "maxcrosspost" have become obsolete in the main configuration file and have to be specified in the filter file instead. The advantage of this is that you can do much finer selection on these criteria now.

To replace, for example, a "maxage = 5" specification in the main configuration file, you should add the following to your filter file:

newsgroups = *
maxage = 5
action = kill

The leafnode programs will issue warnings if they encounter obsolete specifications in the main config file.

3.2 Update from versions < 1.9.3

From version 1.9.3 on, the groupinfo file is sorted in a case-insensitive manner. To update correctly, do a "make update" as root after you have successfully completed "make install". This will re-sort the groupinfo file. The old groupinfo file will be stored as groupinfo.old just in case something goes wrong. You also have to change your main config file manually as described in the last section.

3.3 Update from versions < 1.6

Between leafnode-1.6alpha and leafnode-1.6, the format of the groupinfo file changed and some files moved to other places. To update correctly, do a "make update" as root after you have successfully completed "make install". This will reformat the groupinfo file and move the other files into the correct places. The old groupinfo file will be stored as groupinfo.old just in case something goes wrong. You also have to change your main config file manually as described in the section "Update from versions < 2.0".

4. FILES AND DIRECTORIES

Leafnode puts its files in three separate directories: The spool directory, the library directory, and the binaries directory. All directories can be changed at compile time.

In the spool directory you find the stored news, the active file and some other short-lived configuration file. It defaults to /var/spool/news and can be changed at compilation time. There are some special directories here; see the leafnode(8) man page.

The library directory contains long-lived configuration files. It defaults to /etc/leafnode.

The binaries directory, /usr/local/sbin by default, contains the executable programs applyfilter, texpire, fetchnews and leafnode.

The user directory, /usr/local/bin by default, contains the newsq program.

4.1. The main configuration file

The main configuration file contains settings important for all programs of the leafnode suite. It defaults to /etc/leafnode/config, but several configuration files can be used at once by employing the -F switch of the programs.

The file contains two mandatory and a number of optional parameters.

Mandatory parameters

server = news02.bigprovider.com

You have to specify at least one server (except if you want leafnode to serve as a local server only). Usually, this will be the news server of your provider. You can specify more than one server, and fetchnews will retrieve news from all of them, taking care not to transfer articles multiple times to your machine. Servers will be queried in the order specified in the config file.

expire = 5

This parameter determines how many days threads are kept on your hard disk. texpire will delete whole threads, not just single articles.

Server-specific optional parameters

To configure interaction with the server somewhat, you can change the behaviour of fetchnews by setting several server-specific optional parameters. They have to be specified directly after the corresponding "server" statement. A new server statement in the config file will also allow new optional parameters.

username = myname
password = mypasswd

If your upstream server requires a form of authentication, you can set your username and password here.

timeout = 30

It may happen that, due to a bad connection or other reasons, the server stops talking to you while you fetch news. The "timeout" parameter determines the number of seconds fetchnews is supposed to wait for a response from the server before giving up.

nodesc = 1

Some servers are unable to deliver descriptions of new newsgroups correctly. When fetchnews encounters such a server, it will print the warning

server.name does not process LIST NEWSGROUPS news.group.name correctly: use nodesc

To allow for shorter download times, you should in this case set "nodesc = 1" for that particular server in the configuration file.

port = 8000

Normally, fetchnews will try to retrieve news from port 119 (the standard nntp port) of your upstream server. If the upstream server runs on a different port, you can specify it with this option.

General optional parameters

hostname = host.domain.country

If the postings that you write do not have message IDs (generated by the newsreader), leafnode will generate a message ID for you. (It will never overwrite an already existing message ID.) Message IDs generated by leafnode feature ".ln" before the @ sign. The message ID contains the fully qualified hostname of your machine which may not always be what you want. In that case, you can override the use of the hostname by using the "hostname" option.

create_all_links = 1

Usually fetchnews will store articles only in the newsgroups which you consider interesting. Unfortunately, this makes it difficult to score for the number of newsgroups a message is posted to because the Newsgroups: header is not featured in the overview information; therefore you can determine the number of newsgroups an article is crossposted to only from the Xref: header. If "create_all_links" is set to 1, fetchnews will store articles in all newsgroups which they are posted to, making all these newsgroups turn up in the Xref: header.

maxfetch = 1000

This makes fetchnews fetch at most 1000 articles per newsgroup per server. It is not really sensible (because you won't see all the traffic in a group), but retained for compatibility reasons.

initialfetch = 100

If you subscribe to a new group, fetchnews will try to fetch all articles in that group which are on your upstream server. Obviously, this may cause a lot of traffic if the group is very busy. Therefore you can restrict the maximal number of articles to fetch in the first run with this option.

groupexpire a.news.group = 20

As outlined above, texpire will expire threads after "expire" days. If you want to adjust expiry times for certain groups, you may use the "groupexpire" parameter to do just that. You can specify groups or use wildcards; for example

groupexpire *.announce = 20

will affect expiry times of all groups ending with ".announce".

filterfile = /etc/leafnode/filters

If you want to employ filtering on incoming messages, you have to specify the path where the filterfile can be found. The format of the filterfile is described in the next chapter.

timeout_short = 2
timeout_long = 5

These two parameters determine how quickly a group is unsubscribed by leafnode after you have stopped reading it. If you have looked into the group only once, subscription is stopped after "timeout_short" days; if you have read it more regularly, subscription is stopped after "timeout_long" days. (You can stop subscription immediately by removing the corresponding file in /var/spool/news/interesting.groups/).

timeout_active = 90

By default, fetchnews will re-read active files from the upstream server every 90 days. This interval can be changed by setting "timeout_active" to a different value. Re-reading the active file frequently will keep it a little bit smaller but will increase the on-line time.

delaybody = 1

This option switches fetchnews into a mode where only the headers of articles are fetched for visual inspection. Only the articles read in the newsreader will be retrieved the next time fetchnews is called. This can save a huge amount of disk space and download time but requires more manual intervention on the user side.

debugmode = 1

This option forces the leafnode programs into logging lots of information via the syslog daemon. It is only useful if you want to hunt down bugs.

4.2. The filter file

5. LOCAL NEWSGROUPS

From version 2.0 on, Leafnode is able to handle local newsgroups. Local newsgroups are groups that exist only on your local server but not on upstream servers.

To create a local newsgroup, you have to think of a newsgroup name which should not exist on any of your upstream servers. It is therefore a good idea to start a new top-level hierarchy. You should also make up a description for your newsgroup.

If you choose a newsgroup name which exists already on an upstream server, the newsgroup is not treated as a local one.

Next, you write both the name and description into the file /var/spool/news/leaf.node/local.groups using your preferred text editor. The file should consist of lines in the format

news.group.name description

The first word of each line is taken as the newsgroup name; the rest of the line is interpreted as description. For example, to set up a newsgroup "local.leafnode" which deals with Leafnode's internals, you would put a line

local.leafnode Local leafnode user group

into /var/spool/news/leaf.node/local.groups . (If your newsspool does not reside in /var/spool/news, replace this part of the pathname with the appropriate directory.)

That's it! You just have created a local newsgroup and can now access it using your newsreader of choice.

6. DEINSTALLATION

If you want to uninstall leafnode (e.g. because you want to replace it with another newsserver) and have a Makefile available, you can achieve this by doing "make uninstall". This will remove the executables, newsspool and configuration directories. If you only want to remove the executables and leave the spool and configuration directories untouched, use "make uninstall-bins" instead.

There is no need to uninstall an old version of leafnode before updating.

7. FREQUENTLY ASKED QUESTIONS

See the file FAQ.

8. COPYRIGHT

Major modifications were made by
Randolf Skerka <Randolf.Skerka@gmx.de>
Kent Robotti <robotti@erols.com>
Markus Enzenberger <enz@cip.physik.uni-muenchen.de>
Matthias Andree <mandree@dosis.uni-dortmund.de>
Jörg Dietrich <joerg@dietrich.net>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Redistribution and use in any form are permitted provided that the following restrictions are met:

Source distributions must retain this entire copyright notice and comment.
Binary distributions must include the acknowledgement ``This product includes software developed by Rich Salz'' in the documentation or other materials provided with the distribution. This must not be represented as an endorsement or promotion without specific prior written permission.
The origin of this software must not be misrepresented, either by explicit claim or by omission. Credits must appear in the source and documentation.
Altered versions must be plainly marked as such in the source and documentation and must not be misrepresented as being the original software.
THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.