General Topics

Accessing Logs

Most logs are found in ~/logs/frontend and ~/logs/user.

Note

To prevent log files from becoming too large, logs are rotated daily at 3:00 a.m. server time. Each existing log is renamed by appending a number which indicates how many days old the log is. For example, error_website_name.log is named error_website_name.log.1 initially. Each subsequent day the log is incremented by one, until the log has aged seven days, upon which it is deleted.

Front-end Logs

~/logs/frontend stores logs associated with website entries and shared Apache. There are four types of logs which appear in ~/logs/frontend:

  • Website access logs – Logs of the filename form beginning access_website_name.log record attempts to access a website. Such logs record:
    • originating IP address,
    • date and time,
    • HTTP method used,
    • the specific URL accessed,
    • and the user-agent which made the request.
  • Website error logs – Logs of the filename form beginning error_website_name.log record error events associated with websites. Such logs record a date and time and an error message.
  • Shared Apache access logs – Logs of the filename form beginning access_website_name_php.log record access attempts for all shared Apache-based applications reached through website_name. This includes all shared Apache-based applications, such as Trac, Subversion, and Static/CGI/PHP applications. Such logs record:
    • originating IP address,
    • date and time,
    • HTTP method used,
    • the specific URL accessed,
    • and the user-agent which made the request.
  • Shared Apache error logs – Logs of the filename form beginning error_website_name_php.log record error events for all shared Apache-based applications reached through website_name. This includes all shared Apache-based applications, such as Trac, Subversion, and Static/CGI/PHP applications. Such logs record a date and time and an error message.

For example, suppose you have a Website entry with this configuration:

mysite
    / --> htdocs (Static/CGI/PHP)
    /blog/ --> wp (WordPress)
    /testing/ --> fancyapp (Django)

mysite‘s access logs are stored in files beginning with access_mysite.log, while mysite‘s error logs are stored in files beginning with error_mysite.log. access_mysite_php.log records htdocs and wp‘s errors but not fancyapp‘s errors. fancyapp‘s error logs are stored elsewhere; see User Logs for details.

User Logs

~/logs/user stores logs associated with many applications, including popular applications such as Django. There are two types of logs which appear in ~/logs/user:

  • Access logs – Logs of the filename form access_app_name.log record attempts to access the named application. The format of such logs vary with the type of long-running server process associated with the application, but typically these logs record originating IP address, date and time, and other details.
  • Error logs – Logs of the filename form error_app_name.log record error events associated with the named application. Typically, these logs record error messages and their date and time.

Note

Some older installed applications may not store their logs in ~/logs/user. If you cannot find a particular application’s logs in ~/logs, look in the application’s directory (~/webapps/app_name). For example:

  • Django: ~/webapps/app_name/apache2/logs
  • Rails: ~/webapps/app_name/logs/
  • Zope: ~/webapps/app_name/Zope/var, ~/webapps/app_name/zinstance/var/log

Monitoring Memory Usage

To see a list of processes and how much memory they’re using:

  1. Open an SSH session to your account.
  2. Enter ps -u username -o rss,command, where username is your WebFaction account name and press Enter.

The first column is the resident set size, the amount of memory in use by the process. The second column is the process’s command along with any arguments used to start it.

Repeat these steps as needed for any additional SSH users you have created.

Example Memory Usage

For example, consider a user, johndoe, monitoring his memory usage.

 [johndoe@web100 ~]$ ps -u johndoe -o rss,comm
  RSS COMMAND
 1640 PassengerNginxHelperServer /home/johndoe/webapps/rails/gems/gems/passenger-2.2.8 /home/johndoe/webapps/rails/bin/ruby 3 4 0 6 0 300 1 nobody 4294967295 4294967295 /tmp/passenger.4583
 7676 Passenger spawn server
  544 nginx: master process /home/johndoe/webapps/rails/nginx/sbin/nginx -p /home/johndoe/webapps/rails/nginx/
  844 nginx: worker process
  896 ps -u johndoe -o rss,command
 3564 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
12504 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
 2600 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
23740 /usr/local/apache2-mpm-peruser/bin/httpd -k start
23132 /usr/local/apache2-mpm-peruser/bin/httpd -k start
 1588 sshd: johndoe@pts/1
 1472 -bash

The first row displays the column headers:

RSS COMMAND

RSS stands for resident set size. RSS is the physical memory used by the process in kilobytes (kB). COMMAND is the process’s command along with any arguments used to start it.

The next four rows show a Rails application running with Passenger and nginx:

1640 PassengerNginxHelperServer /home/johndoe/webapps/rails/gems/gems/passenger-2.2.8 /home/johndoe/webapps/rails/bin/ruby 3 4 0 6 0 300 1 nobody 4294967295 4294967295 /tmp/passenger.4583
7676 Passenger spawn server
 544 nginx: master process /home/johndoe/webapps/rails/nginx/sbin/nginx -p /home/johndoe/webapps/rails/nginx/
 844 nginx: worker process

The PassengerNginxHelperServer and Passenger processes are the passenger component of the application that handles, for example, executing Ruby code. The two nginx processes are the web server component of the application, which respond to the incoming HTTP requests. Altogether these processes are consuming 10,704KB or just over 10 megabytes (MB).

The next row is the ps process itself:

896 ps -u johndoe -o rss,command

This is the command that’s checking how much memory is in use.

The next three rows represent a running Django application:

 3564 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
12504 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start
 2600 /home/johndoe/webapps/django/apache2/bin/httpd -f /home/johndoe/webapps/django/apache2/conf/httpd.conf -k start

Although there are three processes, this is just one ordinary Django application. These are the Apache processes used to respond to HTTP requests and to run Django itself. Together these processes are consuming 18,668KB or just over 18MB of memory.

Finally, the last two lines show us johndoe’s connection to the server:

1588 sshd: johndoe@pts/1
1472 -bash

These processes are the SSH service and the Bash prompt, which allow johndoe to interact with the server from afar. They use relatively little memory, 3,060KB or just under 3MB.

In total, johndoe is using less than 32MB of memory, which is well under the limit for his plan, so he’s not at risk of having his processes terminated and having to find ways to reduce his memory consumption.

If johndoe’s processes had exceeded his plan limits by a small amount, he would receive a warning message. If his processes had exceeded his plan limits by a large amount, his processes would be terminated and he would receive a notification.

Reducing Memory Usage

See also

Application specific memory consumption documentation:

Once you’ve identified where your memory is going you can take steps to reduce your overall memory consumption. Typically, you can think of the memory your applications require in terms of base memory and additional memory.

Base memory consumption is the amount of memory a piece of software requires. For example, a large application like Plone 4 can consume over 60 megabytes of memory, even before the first page request, while an idle Ruby interpreter might consume only a few megabytes. Unfortunately, little can be done about base memory consumption. Aside from switching to a different application or modifying the software, some amount of memory must be consumed to simply start and run the software as expected.

The biggest gains in conserving memory typically come from reducing additional memory consumption. Once your software is up and running, varying amounts of additional memory will be consumed to store your data and process requests. Software might consume more or less memory based on factors such as:

  • the number and duration of threads or processes,
  • the size, number, and frequency of database queries,
  • the size or complexity of objects retained in memory, or
  • the total number of concurrent requests or sessions.

Because there are so many possible ways for an application to consume memory, there isn’t a “one size fits all” solution for reducing memory consumption, but there are a number of common strategies:

  • Serve static files out-of-process. For application types which rely on an additional server, such as Django (Apache) and Ruby on Rails (nginx), serve static files, such as style sheets, JavaScript, and images, with a separate Static-only application.
  • Plug memory leaks. If the software you are using contains a memory leak, it will attempt to consume more and more memory without releasing already consumed but unneeded memory. If code under your control is leaking memory, make sure memory is deallocated or references are eliminated when particular objects or data are no longer needed. If some library or executable is leaking memory out of your control, periodically restarting the software may contain the application’s memory consumption.
  • Use fewer threads or processes. If your software relies on multiple threads or processes, try reconfiguring the software to use a more conservative number of them.
  • Complete recommended maintenance activity. Some software may have maintenance steps which, if left unfinished, may cause increased memory consumption or deteriorated performance. For example, Zope’s Data.fs file requires periodic packing.
  • Don’t keep unnecessary data in memory. If certain data is not frequently accessed or is inexpensive to retrieve, try to not keep it in memory. For example, the data associated with an infrequently accessed page may not be needed until the page is actually requested.
  • Avoid making database queries that return too much information. Not only are such queries slower, but the resulting data will consume additional memory. For example, if the result of some query must be filtered, it may be possible for some memory consumption to be eliminated by writing more specific database queries.
  • Profile your memory consumption. There may be tools available which work with your programming language to help you identify how memory is being consumed. For example, Guppy for Python features the heapy memory inspection module and Xdebug is a popular profiler for PHP.

Setting File Permissions

We recommend that you use the getfacl and setfacl command line tools to manage file permissions. These tools allow you to set file permissions with much greater security and granularity than chmod, chown, and chgrp.

Warning

Use caution when granting other users access to your files, directories, and applications. In many cases, granting access to other users affords them the privileges of your own account. This can put you at risk of unauthorized, malicious activity, like deleting files and sending spam on your behalf. To minimize your risk, grant access only to people you trust and make certain they use the same precautions as you do, like choosing strong passwords.

getfacl and setfacl allow you to manage file access control lists (ACLs). Using ACLs to manage permissions lets you grant permissions to specific users, and not all other users, which is a critical security concern in a shared hosting environment.

The most common use case for granting permissions to another user is to grant permissions to another SSH/SFTP user created with the control panel.

See also

To see complete documentation for getfacl or setfacl:

  1. Open an SSH session to your account.
  2. Enter man getfacl or man setfacl and press Enter.

Reviewing Permissions

To review the permissions for a file or directory:

  1. Open an SSH session to your account.
  2. Enter getfacl path, where path is the path to the file or directory see the permissions of, and press Enter.

For example, getfacl /home/demo returns the ACL for user demo‘s home directory:

# file: .
# owner: demo
# group: demo
user::rwx
user:apache:r-x
user:nginx:r-x
group::--x
mask::r-x
other::---

Removing Access to Others (Global Access)

To prohibit read, write, and execute access for all other users (users which are neither yourself or otherwise specified in the ACL):

  1. Open an SSH session to your account.

  2. Enter setfacl -m o::---,default:o::--- path, where path is the path to a file or directory, and press Enter.

    Note

    If path is a directory, then you can also change permissions recursively: enter setfacl -R -m o::---,default:o::--- path and press Enter.

Granting Access to Specific Users

You can grant users access to individual files or directories, or a whole application.

Applications

To grant another user access to an entire application:

  1. Open an SSH session to your account.

  2. Allow the other user account to locate directories that it has access to within your home directory. Enter setfacl -m u:secondary_username:--x $HOME, where secondary_username is the other user’s username, and press Enter.

  3. Remove the other user’s default access to the applications in your $HOME/webapps directory. Enter setfacl -m u:secondary_username:--- $HOME/webapps/* and press Enter.

    Note

    The above command only affects applications that are currently installed. If you create new applications, then run the command again, or run setfacl -m u:secondary_username:--- $HOME/webapps/new_app, where new_app is the name of the new application.

  4. Grant the user read, write, and execute access to the application’s files and directories. Enter setfacl -R -m u:secondary_username:rwx $HOME/webapps/application, where application is the name of the application to which the other user is to have access, and press Enter.

  5. Grant the user read, write, and execute access to any files and directories created in the application in the future. Enter setfacl -R -m d:u:secondary_username:rwx $HOME/webapps/application and press Enter.

  6. Set your account’s group as the owner of any new files in the application’s directory. Enter chmod g+s $HOME/webapps/application and press Enter.

  7. Grant your account full access to files in the application directory, including any files created in the future by the secondary user. Enter setfacl -R -m d:u:primary_username:rwx $HOME/webapps/application and press Enter.

The other user is granted access to the application.

Tip

The secondary can add a convenience symlink from their home directory to the application directory. To create the symlink:

  1. Open an SSH session to the secondary user account.
  2. Enter ln -s /home/primary_username/webapps/application $HOME/application, where primary_username is the name of the account which owns the application, and press Enter.

Files and Directories

To grant another user read, write, or execute access (or combinations thereof) to a single file or directory:

  1. Open an SSH session to your account.

  2. Enter setfacl -m u:username:permissions path, where

    • username is the username of the user to be granted permissions,
    • permissions is a combination of r, w, or, x for read, write, and execute, respectively, and
    • path is the path from the current directory to the desired file,

    and press Enter.

    Additionally, if path is a directory and u is prepended with default: or d:, then any new files created by that or any other user within the directory will default to those permissions.

The other user is granted access to the path specified.

Scheduling Tasks with Cron

You can use Cron to automatically run commands and scripts at specific times.

To review the contents of your crontab:

  1. Open an SSH session to your account.
  2. Enter crontab -l and press Enter.

To edit your crontab:

  1. Open an SSH session to your account.

  2. Enter crontab -e and press Enter. Your crontab will open in your default text editor (as specified by the EDITOR environment variable).

    Note

    vi is the default editor. To use a different editor to modify your crontab, set the EDITOR environment variable. For example, to use the nano text editor, enter EDITOR=nano crontab -e and press Enter.

  3. Make any desired changes to your cron schedule.

    Note

    Cron jobs run under different conditions than scripts run from a shell. When you prepare your cron activity, do not rely on a specific starting directory. Use absolute paths where possible. Likewise, do not rely on environment variables like PATH. Changes to the environment by .bashrc and other “dot” files are unlikely to be available to your cron task.

    Note

    You can receive the output of cron jobs by setting the MAILTO variable, redirecting output to mail, or logging to a file.

    To send the output of all cron jobs to a single email address, set the MAILTO variable. On a new line before any jobs, insert MAILTO=recipient where recipient is the destination address for cron error messages.

    To use MAILTO on CentOS 6 servers (web300 and greater or dweb89 and greater), you must set the sender address. To set the sender address, on a new line before any jobs (for example, immediately after the line containing MAILTO), insert MAILFROM=sender, where sender is the sender address.

    When using MAILTO on CentOS 5 servers (less than web300 or less than dweb89), the sender address is username@servername.webfaction.com, where username is your username and servername is the WebFaction server (for example, web299).

    On all servers, you can redirect the output of a cron job to mail, which allows you to specify job-specific subject lines, recipient addresses, and sender addresses. For example, this cron job runs once an hour and sends all output of sample_command to a recipient with a custom subject line, recipient address, and sender address:

    0 * * * * example_command 2>&1 | mail -s "Example report subject" recipient@example.com -- -f sender@example.com
    

    Instead of email, you can store the output from a cron job by redirecting it to a log file. For example, this cron job records cron is running every 20 minutes to ~/logs/user/cron.log:

    */20 * * * * echo "cron is running" >> $HOME/logs/user/cron.log 2>&1
    

    See also

    Cron Help Guide from Linux Help

  4. Save and close the file.

Troubleshooting

Unfortunately, things don’t always work as planned. Here you will find troubleshooting hints for general errors and problems sometimes seen among WebFaction users.

See also

WebFaction Blog: Debugging tips for your CGI scripts

Error: Site not configured

The error Site not configured has several common causes and solutions:

  • Cause: You recently created a new website record in the control panel.

    Solution: Wait up to five minutes for your website record to be fully configured.

  • Cause: You created a new website record but did not include the current subdomain, such as www.

    Solution: Modify the website record to use the intended subdomain.

  • Cause: You created a new domain entry in the control panel, but did not create a corresponding site website record.

    Solution: Create a site record which references your newly created domain entry.

  • Cause: You accessed a website with the wrong protocol—in other words, a protocol other than the one configured in the website entry. For example, you accessed a website configured for HTTPS via HTTP or vice-versa.

    Resolution: Reenter the URL with the HTTP or HTTPS as the protocol or modify the website record to use the intended protocol.

  • Cause: You attempted to access the site by the website’s IP address.

    Resolution: Accessing websites by IP addresses is not supported. Please use the URL with the domain name selected in the control panel, typically one you supply or of the form username.webfactional.com.

  • Cause: You account has been disabled because of a billing problem, TOS or AUP violation, or some other problem concerning your account.

    Resolution: If WebFaction disables your account, we will contact you via your account contact email address to let you know the reason and corrective action required. If you suspect that your account has been disabled and you have not been contacted, please open a support ticket.

Error: Not Found

The Not Found and There is no application mounted at the root of this domain error appears when you visit a domain at the root URL path (/), but no application is mounted there, but other applications are mounted elsewhere. Some causes for this error include:

  • You recently modified website record to include an application at the root URL path, but opened the URL before your changes were activated in the web server and DNS configuration. Please wait a moment and refresh.
  • You accessed a website with a protocol other than the one configured in the website record. For example, you added an application to the root of a website configured to use HTTPS, but opened an HTTP URL in your browser. Reenter the URL with the correct protocol, or modify your website records so that the application is available with the indented protocol.
  • There is no application mounted at that domain’s root. Please revisit the control panel and add the application to the website record’s root URL path, verifying that root URL path is correct (for example, verify that there are no unexpected characters after /).

Error: 403 Forbidden

A 403 Forbidden error typically occurs when file system permissions prevent the web server from accessing the page or running a script. Please see Setting File Permissions for more information on changing file permissions.

Error: 502 Bad Gateway

A 502 Bad Gateway error occurs when the front-end web server cannot communicate with the application responsible for responding to the request. To resolve this error:

  • Confirm that the application is running. If the application crashed, was terminated, or has not yet started following a system reboot, the application must be restarted to respond to requests. If the application was terminated, you will receive a notification email with additional information.

    To restart the application manually, please see your application type’s documentation or control panel entry for instructions.

    Most stopped applications are automatically restarted during periodic cron jobs. Please see the application’s entry in the control panel for details.

  • Confirm that the application is listening to the correct port. Check your application’s configuration to make sure that it is listening to the port assigned to the application in the control panel.

Error: 504 Gateway Timeout

A 504 Gateway Timeout error occurs when an application takes too long to respond to an incoming request. This error is often caused by an application receiving heavy traffic. This error can also be caused by an application running too slowly; optimizations may be required to avoid the error. For more information, please see the documentation for your application type.