[Previous][Next][Contents][Index]

Maintaining your Collabra Server

his chapter describes some of the tasks you perform to maintain your server, such as:

Managing expiration of articles

 The Netscape Collabra Server stores discussion group articles in the local spool directory. You determine how long you want to store articles on disk. You can specify a default expiration policy for all discussion groups or you can specify a custom expiration policy for a specific discussion group or groups.

How long you store articles in a discussion group depends on the following factors:

Of course, your expiration policy depends on a combination of the above factors. For example, if your server has a relatively small disk capacity, and some discussion groups are never expired, other discussion groups might expire every 2 days or so. Or, if your server is a local messaging system with extra disk space, you might store articles in some discussion groups for 60 days or more.

When the server performs its daily maintenance tasks, it checks the expiration policy and expires articles accordingly.

Viewing expiration policies

To view your current expiration policies:

  1. Choose Expiration|View/Manage Expiration Policies.

  2. You'll see the default expiration policy and any custom expiration policies you've created. For each policy, you'll see:

You can also edit your expiration policies from this form by clicking Edit for the policy you want to edit.

You can Remove custom expiration policies by clicking Remove for the policy you want to remove.

Editing the default expiration policy

The default expiration policy covers all discussion groups that are not covered by a custom policy. By default, the Collabra Server uses the following values for the default expiration policy:

You can edit the default policy and specify your own values.

To edit the default expiration policy:

  1. Choose Expiration|Edit Default Expiration Policy.

    (You can also access the edit form from the View/Manage Expiration Policies form by scrolling to Default Expiration Policy and clicking Edit.)

  2. In the Remember history field, type the humber of days you want articles to remain in the history file. The Remember history attribute does not tell the server when to expire an article. Remember history tells the server how long to "remember" a specific article; that is, how long the message ID remains in the history file--regardless of how long the article actually remains in the spool.

    The history file prevents your server from accepting articles it has already accepted. To prevent duplicate articles, set the remember history attribute from 7 to 14 days. If you set remember history to fewer days, your history file will be smaller, but it increases the likelihood of getting duplicate articles.

    Note: The remember history setting should be equal to or higher than the longest expiration setting.

    See "The history file" for more information about the history file.

  3. To determine how long you want to keep articles on disk, click one of the following options:

    See "Creating a custom expiration policy" for more information about these options.

  4. Click OK.

The Reset button resets any changes to the form that you have not yet submitted.

Creating a custom expiration policy

You can create a custom expiration policy that overrides your default expiration policy.

To create a custom expiration policy:

  1. Choose Expiration|Create a Custom Expiration Policy.

  2. From the pull-down menu, choose the discussion group criteria for which you want to specify the expiration policy:

    And, in the matching field, type the discussion group pattern that you want to match.

  3. Click one of the following options:

    Obey expiration headers. This option will obey any expiration headers that the client sends when posting a message. If you click this option, you also need to specify how long you want to keep articles that are posted without expiration headers.

    Do not specify this option if you want to maintain control over the expiration policy of all articles stored in your spool.

    Override expiration headers. This option will override any expiration headers that the client sends when posting a message. If you click this option, you must specify how long you want to keep articles on disk. In the associated fields, type the minimum and maximum number of days you want to keep articles.

    For example, if you want to purge articles before their expiration headers might specify, you might type 14 for the maximum number of days. If an article has an expiration header of 30 days and your maximum days option is set to 14 days, the server keeps the article for 14 days.

    Never expire articles. This option will never expire articles. Use this option only if you are sure you have the disk capacity required for the discussion groups you want to support.

  4. Click OK.

The Reset button resets any changes to the form that you have not yet submitted.

Modifying a custom expiration policy

To modify a custom expiration policy:

  1. Choose Expiration|View/Manage Expiration Policies.

  2. Find the policy you want to modify and click Edit.

  3. You can now modify any fields on the Create Custom Expiration Policy form. See "Creating a custom expiration policy" for more information.

Removing a custom expiration policy

 To remove a custom expiration policy:

  1. Choose Expiration|View/Manage Expiration Policies.

  2. Find the policy you want to remove and click Remove.

The history file

 The history file contains a list of the articles your server has stored and possibly expired. The history file includes information about the creation time of the article, the message-ID, and the location of the article in the spool. The Collabra Server adds a line for each article it posts to the spool.

Articles might be sent to your Collabra Server from several other news servers. This means that some articles might be sent more than once. In addition, articles might be delayed in their various paths to your server, sometimes by many days. By referencing the history file, the server can decline articles that it has already stored.

For example, you might have an article stored for one week, then a week later a replication site wants to send the same article to your server. The Collabra Server compares the article message ID with the list of message IDs in the history file. If the server has already received the file, it rejects the file from the replication site. (Message IDs are generated using the creation time in seconds since January, 1970, appended to the process identification and the hostname for the computer that created the article.)

Because the history file can get quite large, you can specify a maximum time each article is to be remembered when defining your expiration policies. After the number of days specified, the server will purge old lines from the file during the expiration process.

Specifying recovery tasks

Under normal circumstances, the news.daily program performs the daily tasks for maintaining the Collabra Server, including maintaining the active file, maintaining the history file, expiring articles, and so on.

There might be times, however, when you need to take recovery steps. For example, if the history file becomes corrupted, the expire program will not remove articles that it should.

From the Collabra Server interface (Advanced Tasks|Recovery), you can perform the following recovery tasks:

Some recovery tasks can be performed at any time. Other tasks can be performed only while the server is up or only while the server is down. The following table shows the available recovery tasks and what state the server must be in to perform the task.
Tasks

 Server up

 Server down

Run news.daily

 Yes

 Yes

Run expire

 Yes

 Yes

Renumber active file

 Yes

 No

Update history file

 Yes

 No

Remake active file

 No

 Yes

Rebuild history indexes

 No

 Yes

Remake history file

 No

 Yes

Remake search indexes

 No

 Yes

To perform one of the recovery tasks:

  1. Choose Advanced Tasks|Recovery.

  2. You will see the status of the server (up or down) and a list of the tasks available when the server is up or down.

  3. To perform a task, click the Do It button associated with the task. Each task is described in more detail in the sections that follow.

Running news.daily

 The news.daily program performs the daily tasks for maintaining the Collabra Server. These tasks include producing a status report, removing old articles, maintaining the active file, maintaining the history file, and rotating the log files.

You might want to run news.daily if:

You can run news.daily when the server is up or down.

Note: If you find that running the news.daily program takes a long time, you might want to run this program from outside the Collabra Server administrative interface.

Windows NT. From Windows NT, execute the following command: 

server-root/news-serverid/run-daily.cmd
Unix. From Unix, execute the following command:

server-root/news-serverid/run-daily

Running expire

 The expire program purges articles from your spool and purges references from the history file as indicated by your configuration settings.

You might want to run expire if:

You can run expire when the server is up or down.

Recovering the active file

The active file contains a list of all the discussion groups stored on your server. For each discussion group, the low and high article numbers and a flag indicating the type of group is included. The server is sensitive to the exact format of this file.

If you are having problems with the active file, you should try renumbering the active file before you remake the active file.

Renumbering the active file

The renumber the active file command will renumber the list of discussion groups stored on your server. You might want to renumber the active file if:

You can renumber the active file only when the server is up.

If you are still having problems after renumbering the active file, you might need to remake the active file.

Remaking the active file

The remake the active file command rebuilds the active file based on the contents of your spool directory.

Remaking the active file can take some time if you are supporting a lot of discussion groups. If you have a full Usenet feed, obtain a copy of the active file from your news provider instead of remaking the active file and then renumber the active file.

You can remake the active file only when the server is down.

Warning!
If you remake the active file, most special flag settings will be lost. For example, flag settings indicating moderated discussion groups and discussion groups that can't be posted to locally will be reset to the default settings.

Recovering the history file

 The history file contains a list of the articles your server has stored and possibly expired. The history file includes information about the creation time of the article, the message-ID, and the location of the article in the spool.

If you are having problems with the history file, you should try rebuilding the history indexes before you update the history file or remake the history file.

Rebuilding the history indexes

The rebuild history index command rebuilds the history.dir and history.pag files from the history text file.

You might use this option if:

You can rebuild the history indexes only when the server is down.

If you are still having problems with the history file after rebuilding the indexes, you might need to update (or remake) the history file.

Updating the history file

The update history file command opens every article to rebuild the history index.

You might want to update the history file if:

You can update the history file only when the server is up.

If you are still having problems with the history file after updating the file, you might need to remake the history file.

Remaking the history file

 The remake the history file command rebuilds the history file.

To fully rebuild the history file, this command opens every article file to find the Message-ID, so this process can be very time consuming, depending on the number of discussion groups your server supports. If you are receiving a full Usenet feed the number of articles can be over a hundred thousand per day.

You might need to remake the history file if:

You can remake the history file only when the server is down.

Remaking the search indexes

The remake the search indexes command rebuilds the search indexes.

You might want to rebuild the search indexes if:

You can remake the search indexes only when the server is down.

Using process feeds

You can customize how your Collabra Server handles articles by running programs, known as process feeds, on the articles.

A process feed is an instruction to the server to send all new incoming articles belonging to the specified discussion group to a program. For example, you might use a process feed to create an archive program by saving each article to a tape backup device.

You choose the program to run and the discussion groups on which to run the program. The program you specify is started when the Collabra Server starts. It reads from standard input two pieces of information for each article that is sent to it:

The more programs you run, the greater the load will be on the system.

Note: Netscape recommends that if your feed volume is large, do not run a program on all articles. This could be an expensive process in terms of CPU load, performance, and so on.

Viewing information about process feeds

To view information about your process feeds:

  1. Choose Advanced Tasks|View/Manage Process Feeds. For each process feed, the form shows:

  2. From this form, you can navigate to other forms that enable you to edit or remove process feeds.

Creating a process feed

 To create a process feed:

  1. Choose Advanced Tasks|Create a Process Feed.

  2. In the Process Feed Identifier field, type a unique identifier for this process feed. The identifier must not contain spaces or punctuation marks.

  3. In the Run this process field, type the absolute pathname of the program you want to run on the articles. Also type any command-line arguments you want to pass to the program.

  4. Click one of the following options to specify the type of pathname you want to feed to the process:

  5. Click one of the following options to indicate which discussion groups are sent to the process:

  6. If you are not yet ready to use the process feed, you can disable the process feed by clicking the button: Disable this process feed.

    When you are ready to enable the process feed, deselect the button.

  7. If you want to specify advanced options, click Advanced Specification; otherwise, go to step 8.

    In the Advanced Specification field, you can type discussion group patterns that conform to the INN pattern specification.

  8. Click OK.

The Reset button resets any changes to the form that you have not yet submitted.

Editing process feed settings

To edit your process feed settings:

  1. Choose Advanced Tasks|View/Manage Process Feeds.

  2. Find the process feed you want to edit and click Edit. See "Creating a process feed" for more information about the fields.

  3. Click OK.

The Reset button resets any changes to the form that you have not yet submitted.

Disabling a process feed

To disable a process feed:

  1. Choose Advanced Tasks|View/Manage Process Feeds.

  2. Scroll to the process feed you want to disable and click Edit.

  3. Click the button: Disable this process feed.

  4. When you are ready to enable the process feed, deselect the button.

  5. Click OK.

Removing a process feed

 To remove a process feed:

  1. Choose Advanced Tasks|View/Manage Process Feeds.

  2. Scroll to List of Process Feeds and find the process you want to remove.

  3. Click Remove.

Cancelling articles

 At times, you might want to cancel articles. To cancel an article:

  1. Choose Advanced Tasks|Cancel Article.

  2. In the Message ID field, type the message ID. To determine the message ID, you can specify show all headers in your newsreader process. Here's an example of a message ID: <330BC222.6A9A@royal.com>.

  3. Click OK.

The Reset button resets any changes to the form that you have not yet submitted.

Using the ctlinnd utility

The ctlinnd utility is a standalone program that can communicate with the main Collabra Server process, called innd.

If necessary, you can use ctlinnd to perform basic administration functions of the Netscape Collabra Server. You might want to use ctlinnd, for example, if your browser is unavailable or for scripting purposes.

Application developers might use ctlinnd as an API to the Collabra Server. By using ctlinnd as an API, developers can write programs that control the server.

Caution
The ctlinnd utility does not provide the built-in protection mechanisms that are provided with the web-based administrative forms. If you use ctlinnd, be careful. You should use ctlinnd only if you feel confident about what you are doing.
The following sections describe:

Setting up for Windows NT systems

 The ctlinnd utility is located in the directory bin/news/bin relative to your server root. If you installed your Collabra Server in C:/Netscape/Server, the path to ctlinnd is C:/Netscape/Server/bin/news/bin/ctlinnd. This is the default location.

To run, ctlinnd must know where to find your basic configuration information. This information is stored in the nsnews.conf file. So that ctlinnd will know where to find your configuration information, you must set the environment variable NS_NEWSCONF.

From the Program Manager, open "Control Panel" then open "System." At the bottom are two boxes.

Assume your server is named "foo", so your server is in
C:/Netscape/Server/news-foo.

In the Variable box, type:

NS_NEWSCONF
In the Value box, type:

C:/Netscape/Server/news-foo/config/nsnews.conf
Make any necessary changes based on where you installed your server and what you named it. You need to open a new DOS window to run ctlinnd.

Setting up for Unix systems

The ctlinnd utility is located in the directory bin/news/bin relative to your server root.If you installed your Collabra Server in /usr/ns-home, the path to ctlinnd is /usr/ns-home/bin/news/bin/ctlinnd. This is the default location.

To run, ctlinnd must know where to find your basic configuration information. This information is stored in the nsnews.conf file. So that ctlinnd will know where to find your configuration information, you must set the environment variable NS_NEWSCONF.

How you set the NS_NEWSCONF variable depends on what shell you are running. Assume you named your server "foo". Then your server is in /usr/ns-home/news-foo. At your shell prompt, type:

For csh and derivatives:

setenv NS_NEWSCONF /usr/ns-home
For sh and derivatives:

NS_NEWSCONF=/usr/ns-home/news-foo/config/nsnews.conf
export NS_NEWSCONF
Make any necessary changes based on where you installed your server and what you named it.

Running ctlinnd

 To run ctlinnd, type:

ctlinnd command [arguments]
For example, to remove the discussion group music.beatles.paul, type:

ctlinnd rmgroup music.beatles.paul

ctlinnd commands

 Table 8.2 summarizes the ctlinnd commands. Each command is described in more detail following the table.
Command

 Description

cancel

 Cancels a posting.

go

 Restarts a server that has been paused or throttled.

import

 Imports an active file into the server.

mode

 Displays the current status of the server.

newgroup

 Creates a new discussion group.

pause

 Pauses the server; no new articles are accepted. Articles are not rejected; the server attempts to post when the pause is lifted. The server is "hung" until the pause is lifted.

reload

 Reloads a copy of the configuration file.

renumber

 Forces the server to scan the spool at the specified discussion group and verify the article numbers listed in the active file.

rmgroup

 Removes the specified discussion group.

shutdown

 Shuts down the server.

throttle

 Throttles the server; no new articles are accepted. Articles are rejected immediately. The server does not remain in a "hung" state.

cancel

The cancel command removes a specific posting from your Collabra Server. 

cancel message-id
This command does not send out a cancel message to other servers. It only cancels it locally.

You can find the message-ID by looking at the message headers in your newsreader. The header line begins with Message-ID:. If the message-ID has any odd characters, you may need to enclose the message-ID parameter in quotes.

go

The go command starts a server that has been paused or throttled for some reason. 

go reason
The reason parameter must be the same that was used to pause or throttle the server.

To view the current state and reason, use the mode command. You should use this command only if you are sure that the reasons for pausing or throttling the server have been fixed.

import

The import command imports an active file into the server.

import file pattern creator
The file parameter must specify the full path to an existing active file in proper format.

The pattern parameter specifies which discussion groups to import; this parameter must me enclosed in quotes. For example, `alt.*' specifies that you want to import all of the alt discussion groups (that you do not already have) from the active file. To specify all groups in the active file, type `*'.

The creator parameter is used for record purposes only. Typically, creator is an email address.

mode

The mode command displays the current status of the server, including the reason it may be paused or throttled and a count of the current number of clients.

mode

newgroup

The newgroup command creates a new discussion group on the Collabra Server only. This command does not send a newgroup control message to other servers. 

newgroup groupname rest creator
The groupname parameter must be a proper discussion group name.

The rest parameter may be a y, n, or m, indicating a postable, non-postable, or moderated discussion group, respectively.

The creator parameter should be the name of the person running this command. Typically, it is the user's email address.

pause

The pause command pauses the Collabra Server so that it will not accept new articles. Articles are not rejected; the server attempts to post when the pause is lifted.

pause reason

Readers are not disconnected, but the server is in a "hung" state until the pause is lifted.

For logging purposes you must provide a brief reason for pausing the server. You will use this reason when specifying the go command.

reload

The reload command reloads a copy of a configuration file. You must provide the short name of the file to reload and a brief reason for reloading the file. 

reload filename reason

renumber

The renumber command tells the Collabra Server to look through the spool at the specified discussion group and verify the article numbers listed in the active file. If the discussion group name is " or "" it will do this for all discussion groups. 

renumber discussion-group-name

rmgroup

The rmgroup command removes the specified discussion group from the Collabra Server only. No rmgroup control messages are sent to other servers. Only the active file is modified. Articles are left in the spool until they expire.

rmgroup group

shutdown

 The shutdown command shuts down the Collabra Server. 

shutdown

throttle

 The throttle command throttles the server; no new articles are accepted. Articles are rejected immediately. The server does not remain in a hung state.

For logging purposes you must provide a brief reason for throttling the server. You will use this reason when specifying the go command.

Using ctlinnd on scripts and programs

You can use ctlinnd to write scripts to stop the server at some regular interval, monitor server status, cancel messages, and so on.

Because ctlinnd is a standalone program, it can be run from a shell script, a perl script, or even from another executable. Below are three examples of running ctlinnd. Each program takes a single groupname as an argument and removes that group from the server.

More complicated scripts follow the same pattern. The examples show Unix pathnames. If you are running ctlinnd on Windows NT, remember to use DOS file paths.

Bourne shell. To use from the Bourne shell (/bin/sh):

#!/bin/sh
group=$1
NS_NEWSCONF=/usr/ns-home/news-foo/config/nsnews.conf
export NS_NEWSCONF
echo "Removing group $group"
/usr/ns-home/bin/news/bin/ctlinnd rmgroup $group
perl script. To use from a perl script:

#!/usr/local/bin/perl
$group=$ARGV[0];
$ENV{'NS_NEWSCONF'}=/usr/ns-home/news-foo/config/nsnews.conf
print "Removing group $groupn";
system("/usr/ns-home/bin/news/bin/ctlinnd rmgroup $group");
C code. To use from C code:

void main(int argc, char **argv) {
          char                     s[1024];
          char                     *group;

          group = argv[1];
          putenv("NS_NEWSCONF=/usr/ns-home/news-foo/config/nsnews.conf");
          sprintf(s,""/usr/ns-home/bin/news/bin/ctlinnd rmgroup %s",group);
          printf("Removing group %sn",group);
          system(s);
}

Troubleshooting ctlinnd

 If you have problems when running ctlinnd, check the following list of symptoms and probable causes.

ctlinnd core-dumps/bus-errors/seg-faults/and so on whenever I try to run it! Why?

Check the value of NS_NEWSCONF before you run ctlinnd. Is it correct? Does it point to the nsnews.conf file? If not, the above problems will occur.

ctlinnd keeps complaining about bad or wrong number of arguments! Why?

Are you providing the correct number of arguments? If you are running ctlinnd from a shell or DOS window, are you using special characters (such as *, /, <, ")? Try surrounding the argument with ' ' or " " to stop expansion of the wildcard at the shell level.

ctlinnd complains about permissions and owner! Why?

Are you logged in as the same user the server is running as? Or are you root (UNIX) or Administrator (NT)? To communicate with the Collabra Server, you must have sufficient privileges. On UNIX, you can su to the news user or to root. On NT, you will have to log in as the news user or Administrator.

I don't see my problem listed here! What do I do?

Post a message to snews://secnews.netscape.com/netscape.server.news with any SSL capable newsreader explaining your problem. In your message, include the server version, what operating system you are running, what kind of hardware, and so on. If possible, include a cut-and-paste of a command session or a copy of your script.

[Previous][Next][Contents][Index]

Copyright © 1997 Netscape Communications Corporation