General Topics

Accessing Logs

Most logs are found in ~/logs/frontend and ~/logs/user.

Note

To prevent log files from becoming too large, logs are rotated daily at 3:00 a.m. server time. Each existing log is renamed by appending a number which indicates how many days old the log is. For example, error_website_name.log is named error_website_name.log.1 initially. Each subsequent day the log is incremented by one, until the log has aged seven days, upon which it is deleted.

Front-end Logs

~/logs/frontend stores logs associated with website entries and shared Apache. There are four types of logs which appear in ~/logs/frontend:

• Website access logs – Logs of the filename form beginning access_website_name.log record attempts to access a website. Such logs record:
  • originating IP address,
  • date and time,
  • HTTP method used,
  • the specific URL accessed,
  • and the user-agent which made the request.
• Website error logs – Logs of the filename form beginning error_website_name.log record error events associated with websites. Such logs record a date and time and an error message.
• Shared Apache access logs – Logs of the filename form beginning access_website_name_php.log record access attempts for all shared Apache-based applications reached through website_name. This includes all shared Apache-based applications, such as Trac, Subversion, and Static/CGI/PHP applications. Such logs record:
  • originating IP address,
  • date and time,
  • HTTP method used,
  • the specific URL accessed,
  • and the user-agent which made the request.
• Shared Apache error logs – Logs of the filename form beginning error_website_name_php.log record error events for all shared Apache-based applications reached through website_name. This includes all shared Apache-based applications, such as Trac, Subversion, and Static/CGI/PHP applications. Such logs record a date and time and an error message.

For example, suppose you have a Website entry with this configuration:

mysite
    / --> htdocs (Static/CGI/PHP)
    /blog/ --> wp (WordPress)
    /testing/ --> fancyapp (Django)

mysite‘s access logs are stored in files beginning with access_mysite.log, while mysite‘s error logs are stored in files beginning with error_mysite.log. access_mysite_php.log records htdocs and wp‘s errors but not fancyapp‘s errors. fancyapp‘s error logs are stored elsewhere; see User Logs for details.

User Logs

~/logs/user stores logs associated with many applications, including popular applications such as Django. There are two types of logs which appear in ~/logs/user:

• Access logs – Logs of the filename form access_app_name.log record attempts to access the named application. The format of such logs vary with the type of long-running server process associated with the application, but typically these logs record originating IP address, date and time, and other details.
• Error logs – Logs of the filename form error_app_name.log record error events associated with the named application. Typically, these logs record error messages and their date and time.

Note

Some older installed applications may not store their logs in ~/logs/user. If you cannot find a particular application’s logs in ~/logs, look in the application’s directory (~/webapps/app_name). For example:

• Django: ~/webapps/app_name/apache2/logs
• Rails: ~/webapps/app_name/logs/
• Zope: ~/webapps/app_name/Zope/var, ~/webapps/app_name/zinstance/var/log

Monitoring Memory Usage

To see a list of processes and how much memory they’re using:

1. Open an SSH session to your account.
2. Enter ps -u username -o rss,command, where username is your WebFaction account name and press Enter.

The first column is the resident set size, the amount of memory in use by the process. The second column is the process’s command along with any arguments used to start it.

Repeat these steps as needed for any additional SSH users you have created.

Example Memory Usage

For example, consider a user, johndoe, monitoring his memory usage.

 [johndoe@web100 ~]$ ps -u johndoe -o rss,comm
  RSS COMMAND
 1640 PassengerNginxHelperServer /home/johndoe/webapps/rails/gems/gems/passenger-2.2.8 /home/johndoe/webapps/rails/bin/ruby 3 4 0 6 0 300 1 nobody 4294967295 4294967295 /tmp/passenger.4583
 7676 Passenger spawn server
  544 nginx: master process /home/johndoe/webapps/rails/nginx/sbin/nginx -p /home/johndoe/webapps/rails/nginx/
  844 nginx: worker process
  896 ps -u johndoe -o rss,command
 3564 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
12504 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
 2600 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
23740 /usr/local/apache2-mpm-peruser/bin/httpd -k start
23132 /usr/local/apache2-mpm-peruser/bin/httpd -k start
 1588 sshd: johndoe@pts/1
 1472 -bash

The first row displays the column headers:

RSS COMMAND

RSS stands for resident set size. RSS is the physical memory used by the process in kilobytes (kB). COMMAND is the process’s command along with any arguments used to start it.

The next four rows show a Rails application running with Passenger and nginx:

1640 PassengerNginxHelperServer /home/johndoe/webapps/rails/gems/gems/passenger-2.2.8 /home/johndoe/webapps/rails/bin/ruby 3 4 0 6 0 300 1 nobody 4294967295 4294967295 /tmp/passenger.4583
7676 Passenger spawn server
 544 nginx: master process /home/johndoe/webapps/rails/nginx/sbin/nginx -p /home/johndoe/webapps/rails/nginx/
 844 nginx: worker process

The PassengerNginxHelperServer and Passenger processes are the passenger component of the application that handles, for example, executing Ruby code. The two nginx processes are the web server component of the application, which respond to the incoming HTTP requests. Altogether these processes are consuming 10,704KB or just over 10 megabytes (MB).

The next row is the ps process itself:

896 ps -u johndoe -o rss,command

This is the command that’s checking how much memory is in use.

The next three rows represent a running Django application:

 3564 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
12504 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
 2600 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start

Although there are three processes, this is just one ordinary Django application. These are the Apache processes used to respond to HTTP requests and to run Django itself. Together these processes are consuming 18,668KB or just over 18MB of memory.

Finally, the last two lines show us johndoe’s connection to the server:

1588 sshd: johndoe@pts/1
1472 -bash

These processes are the SSH service and the Bash prompt, which allow johndoe to interact with the server from afar. They use relatively little memory, 3,060KB or just under 3MB.

In total, johndoe is using less than 32MB of memory, which is well under the limit for his plan, so he’s not at risk of having his processes terminated and having to find ways to reduce his memory consumption.

If johndoe’s processes had exceeded his plan limits by a small amount, he would receive a warning message. If his processes had exceeded his plan limits by a large amount, his processes would be terminated and he would receive a notification.

Reducing Memory Usage

See also

Application specific memory consumption documentation:

• Django
• mod_wsgi

Once you’ve identified where your memory is going you can take steps to reduce your overall memory consumption. Typically, you can think of the memory your applications require in terms of base memory and additional memory.

Base memory consumption is the amount of memory a piece of software requires. For example, a large application like Plone 4 can consume over 60 megabytes of memory, even before the first page request, while an idle Ruby interpreter might consume only a few megabytes. Unfortunately, little can be done about base memory consumption. Aside from switching to a different application or modifying the software, some amount of memory must be consumed to simply start and run the software as expected.

The biggest gains in conserving memory typically come from reducing additional memory consumption. Once your software is up and running, varying amounts of additional memory will be consumed to store your data and process requests. Software might consume more or less memory based on factors such as:

• the number and duration of threads or processes,
• the size, number, and frequency of database queries,
• the size or complexity of objects retained in memory, or
• the total number of concurrent requests or sessions.

Because there are so many possible ways for an application to consume memory, there isn’t a “one size fits all” solution for reducing memory consumption, but there are a number of common strategies:

• Serve static files out-of-process. For application types which rely on an additional server, such as Django (Apache) and Ruby on Rails (nginx), serve static files, such as style sheets, JavaScript, and images, with a separate Static-only application.
• Plug memory leaks. If the software you are using contains a memory leak, it will attempt to consume more and more memory without releasing already consumed but unneeded memory. If code under your control is leaking memory, make sure memory is deallocated or references are eliminated when particular objects or data are no longer needed. If some library or executable is leaking memory out of your control, periodically restarting the software may contain the application’s memory consumption.
• Use fewer threads or processes. If your software relies on multiple threads or processes, try reconfiguring the software to use a more conservative number of them.
• Complete recommended maintenance activity. Some software may have maintenance steps which, if left unfinished, may cause increased memory consumption or deteriorated performance. For example, Zope’s Data.fs file requires periodic packing.
• Don’t keep unnecessary data in memory. If certain data is not frequently accessed or is inexpensive to retrieve, try to not keep it in memory. For example, the data associated with an infrequently accessed page may not be needed until the page is actually requested.
• Avoid making database queries that return too much information. Not only are such queries slower, but the resulting data will consume additional memory. For example, if the result of some query must be filtered, it may be possible for some memory consumption to be eliminated by writing more specific database queries.
• Profile your memory consumption. There may be tools available which work with your programming language to help you identify how memory is being consumed. For example, Guppy for Python features the heapy memory inspection module and Xdebug is a popular profiler for PHP.

Setting File Permissions

We recommend that you use the getfacl and setfacl command line tools to manage file permissions. These tools allow you to set file permissions with much greater security and granularity than chmod, chown, and chgrp.

getfacl and setfacl allow you to manage file ACLs. Using ACLs to manage permissions lets you grant permissions to other users without granting those permissions to all other users, which is a critical security concern in a shared hosting environment.

The most common use case for granting permissions to another user is to grant permissions to the apache user or another SSH/SFTP user created with the control panel.

See also

You can view complete documentation for getfacl or setfacl by entering man getfacl or man setfacl, respectively, during an interactive SSH session.

Reviewing Permissions

To review the permissions granted to a given file or directory, enter getfacl path where path is the path from the current directory to the desired file or directory.

For example, getfacl ran in demo‘s home directory returns:

# file: .
# owner: demo
# group: demo
user::rwx
user:apache:r-x
user:nginx:r-x
group::--x
mask::r-x
other::---

If path is a directory, you can review permissions recursively by adding -R immediately after getfacl.

Removing Access to Others (Global Access)

To prohibit read, write, and execute access for all other users (users which are neither yourself or otherwise specified in the ACL) enter setfacl -m o::---,default:o::--- path where path is the path to a file.

If path is a directory, you can set permissions recursively by adding -R immediately after setfacl.

Granting Access to Specific Users

To grant specific users read, write, and execute access, enter setfacl -m u:username:permissions path where

• username is the username of the user to grant permissions,
• permissions is a combination r, w, and, x for read, write, and execute,
• and path is the path from the current directory to the desired file.

Additionally, if u is prepended with default: or d: and path is a directory, any new files created by that or any other user within path will default to those permissions.

For example, suppose you would like to grant one of your other SSH accounts access to an application in a subdirectory of your ~/webapps directory. To grant the other user account access:

1. Log in to an SSH session with your account name (the username you use to log in to the control panel).

2. Enter setfacl -m u:secondary_username:--x $HOME, where secondary_username is the other user account name, and press Enter. The command permits the other account to locate directories that it has access to within your home directory.

3. Enter setfacl -m u:secondary_username:--- $HOME/webapps/* and press Enter. This command disallows the other account to access the applications in your ~/webapps directory.

   Note

   The above command only affects applications that are currently installed. If you create new applications, you will need to run that command again, or secure the application individually with setfacl -m u:secondary_username:--- $HOME/webapps/name_of_new_app.

4. Enter setfacl -R -m u:secondary_username:rwx $HOME/webapps/application, where application is the name of the application to which the other user is to have access, and press Enter. This command grants the other user read, write, and execute access to all files and directories within the application.

5. Enter setfacl -R -m d:u:secondary_username:rwx $HOME/webapps/application and press Enter. This command makes all new files in the application directory and its subdirectories have the same permissions by default.

6. Enter chmod g+s $HOME/webapps/application and press Enter. This command makes new files in the directory owned by the main account’s group.

7. Enter setfacl -R -m d:u:primary_username:rwx $HOME/webapps/application and press Enter. This command allows the primary user to continue to have full access to files, even if they’re created by the secondary user.

Now the secondary user can read, write, and execute files within the web app’s directory. Likewise, the primary user continues to have full access to the same files.

The secondary user can add a convenient symlink to the web app in their home directory: enter ln -s /home/primary_username/webapps/application ~/application, where primary_username is the name of the account used to log in to the control panel, and press Enter.

Scheduling Tasks with Cron

You can use Cron to automatically run commands and scripts at specific times.

To review the contents of your crontab:

1. Open an SSH session.
2. Enter crontab -l and press Enter.

To edit your crontab:

1. Open an SSH session.

2. Enter crontab -e and press Enter. Your crontab will open in your default text editor (as specified by the EDITOR environment variable).

   Note

   vi is the default editor. To use a different editor with crontab, you can use the EDITOR environment variable. For example, to use the nano text editor with crontab, enter EDITOR=nano crontab -e and press Enter.

3. Make any desired changes to your cron schedule.

   Note

   Cron jobs run under different conditions than scripts run from a shell. When you prepare your cron activity, do not rely on a specific starting directory: use absolute paths where possible. Likewise, do not rely on environment variables like PATH: changes to the environment by .bashrc and other “dot” files are unlikely to be available to your cron task.

   Note

   You can send cron output to email with the MAILTO directive.

   To specify an address to receive messages, add MAILTO=address to a new line at the top of your crontab, where address is the destination address for cron error messages.

   By default the sender is username@servername.webfaction.com, where username is your username and servername is the WebFaction server (for example, web300).

   On CentOS 6 servers (Web300 and greater), you can change the sender’s address with the MAILFROM variable. To change the sender’s address, insert a new line at the top of your crontab containing MAILFROM=address where address is the sending address.

   On all servers, you can redirect the output of a cron job to mail that allows you to specify job-specific subject lines, recipient addresses, and sender addresses. For example, this cron job runs once an hour and mails all output of sample_command to a recipient with a custom subject line, recipient address, and sender address.

   0 * * * * example_command 2>&1 | mail -s "Example report subject" recipient@example.com -- -f sender@example.com
   

   Instead of email, you may also store the output from a cron job by redirecting it to a log file. For example, this cron job would record cron is running every 20 minutes to ~/cron.log:

   */20 * * * * echo "cron is running" > $HOME/cron.log 2>&1
   

   See also

   Cron Help Guide from Linux Help

4. Save and close the file.

Troubleshooting

Unfortunately, things don’t always work as planned. Here you will find troubleshooting hints for general errors and problems sometimes seen among WebFaction users.

See also

WebFaction Blog: Debugging tips for your CGI scripts

Error: Site not configured

The error Site not configured has several common causes and solutions:

• Cause: You recently created a new website record in the control panel.

  Solution: Wait up to five minutes for your website record to be fully configured.

• Cause: You created a new website record but did not include the current subdomain, such as www.

  Solution: Modify the website record to use the intended subdomain.

• Cause: You created a new domain entry in the control panel, but did not create a corresponding site website record.

  Solution: Create a site record which references your newly created domain entry.

• Cause: You accessed a website with the wrong protocol—in other words, a protocol other than the one configured in the website entry. For example, you accessed a website configured for HTTPS via HTTP or vice-versa.

  Resolution: Reenter the URL with the HTTP or HTTPS as the protocol or modify the website record to use the intended protocol.

  See also

  WebFaction User Guide > Secure Sites (HTTPS)

• Cause: You attempted to access the site by the website’s IP address.

  Resolution: Accessing websites by IP addresses is not supported. Please use the URL with the domain name selected in the control panel, typically one you supply or of the form username.webfactional.com.

• Cause: You account has been disabled because of a billing problem, TOS or AUP violation, or some other problem concerning your account.

  Resolution: If WebFaction disables your account, we will contact you via your account contact email address to let you know the reason and corrective action required. If you suspect that your account has been disabled and you have not been contacted, please open a support ticket.

Error: Not Found

The Not Found and There is no application mounted at the root of this domain error appears when you visit a domain at the root URL path (/), but no application is mounted there, but other applications are mounted elsewhere. Some causes for this error include:

• You recently modified website record to include an application at the root URL path, but opened the URL before your changes were activated in the web server and DNS configuration. Please wait a moment and refresh.
• You accessed a website with a protocol other than the one configured in the website record. For example, you added an application to the root of a website configured to use HTTPS, but opened an HTTP URL in your browser. Reenter the URL with the correct protocol, or modify your website records so that the application is available with the indented protocol.
• There is no application mounted at that domain’s root. Please revisit the control panel and add the application to the website record’s root URL path, verifying that root URL path is correct (for example, verify that there are no unexpected characters after /).

Error: 403 Forbidden

A 403 Forbidden error typically occurs when file system permissions prevent the web server from accessing the page or running a script. Please see Setting File Permissions for more information on changing file permissions.

Error: 502 Bad Gateway

A 502 Bad Gateway error occurs when the front-end web server cannot communicate with the application responsible for responding to the request. To resolve this error:

• Confirm that the application is running. If the application crashed, was terminated, or has not yet started following a system reboot, the application must be restarted to respond to requests. If the application was terminated, you will receive a notification email with additional information.

  To restart the application manually, please see your application type’s documentation or control panel entry for instructions.

  Most stopped applications are automatically restarted during periodic cron jobs. Please see the application’s entry in the control panel for details.

• Confirm that the application is listening to the correct port. Check your application’s configuration to make sure that it is listening to the port assigned to the application in the control panel.

Error: 504 Gateway Timeout

A 504 Gateway Timeout error occurs when an application takes too long to respond to an incoming request. This error is often caused by an application receiving heavy traffic. This error can also be caused by an application running too slowly; optimizations may be required to avoid the error. For more information, please see the documentation for your application type.