This chapter explains how Helix Universal Proxy records information about client connections and other events. Using the log files, you can compile reports about system activity, gathering the statistical information you need.
| Tip: If you're interested in designing custom reports to track specific activities on Helix Universal Proxy, refer to Chapter 14. |
Helix Universal Proxy maintains an access log that includes statistics about client connections. It keeps another log of error and informational messages about Helix Universal Proxy operation. The log files are text files that you can open with any text editor, or parse with a script or application. As accesses or errors occur, Helix Universal Proxy appends information to the end of the appropriate log file. The following sections introduce you to the log files and their features.
The access log records information about requests by RealNetworks media players, Windows Media Player, and QuickTime Player. Using these logs, you can find out what clips were played, the times when media players connected, and so on. This information can help you determine which clips are most popular, for example.
The default access log is proxy.log, which is located in the Logs subdirectory of
the main Helix Universal Proxy installation directory.
Helix Universal Proxy provides six logging styles that determine the amount of information gathered on each access attempt. In general, each style builds on the preceding style, adding more information. For instance, logging style 0 gathers the least amount of information. Logging style 1 includes the style 0 information, and adds more information, and so forth. You choose just one logging style for the entire log.
| For More Information: The section "Access Log File Format" explains the logging styles and information fields. |
All logging styles can record statistics about a media player's playback experience. These statistics let you learn how many media packets were dropped, for instance, or whether the viewer paused the clip. There are four types of client statistics. You can use any combination of these statistics types, up to all four. Or, you can turn off client statistics gathering entirely. As well, users may choose not to report statistics.
| For More Information: See "Client Statistics". |
The error log contains information and error messages about Helix Universal
Proxy operation. By looking for patterns of errors, you can troubleshoot and
correct possible problems on your site. The default file name is proxyerr.log,
and the file is generated in the Logs subdirectory of the main Helix Universal
Proxy installation directory. Helix Universal Proxy records an entry in this log
only when an error occurs. Until an error happens, the file does not exist. The
error log uses the following syntax:
***date time proxyname(process_ID): error_message |
The following table explains these fields in the error log file.
| Note: If you receive a message that refers to a fatal error, contact the RealNetworks Technical Support Department for assistance. |
The access and error log files can grow indefinitely as they accumulate data. To keep log files manageable, you can limit a log file to a specific size. With the access log, you can also create a new log at a preset interval, such as every six hours or two weeks, depending on the amount of data you expect to log. Helix Universal Proxy begins, or rolls, a new log file when the limit is reached. Rolled log files are named with the following format:
|
The name and extension are set through Helix Administrator, as described in "Customizing the Access and Error Logs". The timestamp has the following format, using a 24-hour clock:
YYYYMMDDHHMMSS |
For example, the following file was created on June 22, 2002, at 1:49.53 P.M:
proxy.log.20020622134953 |
Helix Universal Proxy records each access request in a separate record written to a new line in the access log. Fields within a record are separated by spaces or by pipes (|). One record is created for every clip served. If the client requests a presentation that includes several clips, one record is created for each clip in the presentation.
Helix Universal Proxy provides six logging styles, numbered 0 through 5.
Styles 1 through 4 each include the information of lower logging styles. For
example, style 3 collects the same information as styles 0, 1, and 2, as well as
some additional information. The default is style 5, which adds a
presentation_ID field to the information in style 2. The following sections
describe which fields each logging style collects. The section "Access Log
Fields" explains the information logged in each field.
| Tip: Although square brackets in syntax typically indicate optional material, the square brackets shown in the following access log syntax actually appear in the access log records. |
Note:
In the following examples, client statistics are not logged,
so each entry shows [UNKNOWN] where the statistics fields
would be. If you collect client statistics, therefore, each log
entry will contain additional information. For more
information, see "Client Statistics".
|
Logging style 0 uses this format:
client_address - - [timestamp] "GET filename protocol/version" HTTP_status_code |
Here is an example of an actual log record, showing that 858,636 bytes of the requested clip were sent over RTSP:
207.188.7.125 - - [26/Jun/2002:10:31:44 -0700] "GET real9video.rm RTSP/1.0" |
Logging style 1 follows this format:
client_address - - [timestamp] "GET filename protocol/version" HTTP_status_code |
The following sample log record shows the same information as logging style 0, but adds information on file size, clip timeline length, actual time streamed, and resent packages:
207.188.7.125 - - [26/Jun/2002:10:06:33 -0700] "GET real9video.rm RTSP/1.0" |
This is the format for logging style 2, which is identical to style 1, except that it records a global client ID.
client_address - - [timestamp] "GET filename protocol/version" HTTP_status_code |
207.188.7.125 - - [26/Jun/2002:10:07:42 -0700] "GET real9video.rm RTSP/1.0" |
Logging style 3 follows this format. It builds on style 2 by adding information about the streams and the Helix Universal Server or parent Helix Universal Proxy that delivered the clip:
client_address - - [timestamp] "GET filename protocol/version" HTTP_status_code |
This example shows the origin server and stream information added to the end of the record:
207.188.7.125 - - [26/Jun/2002:10:09:09 -0700] "GET real9video.rm RTSP/1.0" |
Logging style 4 adds information about the clip's average bit rate and number of packets sent:
client_address - - [timestamp] "GET filename protocol/version" HTTP_status_code |
207.188.7.125 - - [26/Jun/2002:10:10:04 -0700] "GET real9video.rm RTSP/1.0" |
Logging style 5, which is the default style, does not build on the preceding styles. Instead, it copies style 2 and adds a presentation ID that helps you keep track of presentations that contain multiple clips:
client_address - - [timestamp] "GET filename protocol/version" HTTP_status_code |
The following is an example of a logging style 5 entry:
207.188.7.125 - - [26/Jun/2002:10:11:03 -0700] "GET real9video.rm RTSP/1.0" |
The following table summarizes the various logging fields that may appear in an access record, and indicates which logging styles include the fields. The following sections describe the access log fields in detail.
| Log Field | Logging Styles | Reference |
|---|---|---|
client_address |
0, 1, 2, 3, 4, 5 |
click here |
[timestamp] |
0, 1, 2, 3, 4, 5 |
click here |
"GET filename protocol/version" |
0, 1, 2, 3, 4, 5 |
click here |
HTTP_status_code |
0, 1, 2, 3, 4, 5 |
click here |
bytes_sent |
0, 1, 2, 3, 4, 5 |
click here |
[client_info] |
0, 1, 2, 3, 4, 5 |
click here |
[client_ID] |
2, 3, 4, 5 |
click here |
[client_stats_results] |
1, 2, 3, 4, 5 |
click here |
file_size |
1, 2, 3, 4, 5 |
click here |
file_time |
1, 2, 3, 4, 5 |
click here |
sent_time |
1, 2, 3, 4, 5 |
click here |
resends |
1, 2, 3, 4, 5 |
click here |
failed_resends |
1, 2, 3, 4, 5 |
click here |
[stream_components] |
3, 4 |
click here |
[start_time] |
3, 4 |
click here |
server_address |
3, 4 |
click here |
average_bitrate |
4 |
click here |
packets_sent |
4 |
click here |
presentation_ID |
5 |
click here |
proxy_info |
1, 2, 3, 4, 5 |
click here |
The client_address field gives the IP address of the client, such as 123.45.123.45.
Following the IP address are two hyphens for compatibility with standard Web
server log formats.
The [timestamp] field indicates the time that the client accessed the file
according to the Helix Universal Proxy clock. It uses the following format:
[dd/Mmm/yyyy:hh:mm:ss TZ] |
Here, TZ is the time zone expressed as the number of hours relative to the
Coordinated Universal Time (Greenwich, England). For example:
[26/Jun/2002:10:10:04 -0700] |
The "GET filename protocol/version" field lists the file name and path requested
by the client. The path is everything in the URL after the port number. If the
client requests a file that doesn't exist, UNKNOWN appears in place of the file
name. Possible values for the application-layer protocol used to send the clip
to the client are RTSP, PNA, and MMS. In addition, a letter at the end of the string
indicates which transport type was used:
(blank) |
UDP connection |
T |
TCP connection |
M |
Multicast |
For example, RTSPT means that the clip was streamed using the RTSP protocol
over a TCP connection. The version number indicates the edition of the
protocol.
| For More Information: See "GET Statements". |
The HTTP_status_code field holds a return code that uses the HTTP standard
error codes. It usually returns 200.
The bytes_sent field records the number of bytes transferred to the client by
any type of proxy delivery: pass-through, pull-split or cache mode.
The [client_info] field describes the version and type of client being used.
For RealNetworks clients, [client_info] uses the following format:
|
The following information is recorded:
[WinNT_5.0_6.0.10.714_RealPlayer_RN92PD_en_686] |
Note:
RealAudio Player 1 logs just two fields for [client_info].
They are platform and client.
|
For Windows Media Player, the [client_info] field records the player version
like this:
[NSPlayer/7.1.0.3055] |
For QuickTime Player, the client information records the player version and the operating system. For example:
[QTS (qtver=5.0.2;os=Windows NT 5.0)] |
If client information can't be gathered because the request came from a client
that chose not to send statistics, [UNKNOWN] appears in the [client_info] field.
For [client_ID], the access log can record an identification number for each
media player. This is a globally unique ID. (Helix Universal Proxy typically
resides behind a firewall, therefore the proxy does not attempt to gather
cookie-based ids for clients.) The following sections explain the possible field
entries. The default settings for Helix Universal Proxy and RealNetworks
clients record a global ID for each client access attempt. Users can control
whether GUIDs are transmitted, however. As well, you can disable the logging
of GUIDs through Helix Universal Proxy regardless of client configurations.
| For More Information: See "Modifying the Access Log" for instructions on turning off client ID logging. |
If a RealNetworks client is configured to send a globally unique ID, it does so. For privacy protection, however, RealOne Player is set by default not to send a GUID. Because sending a GUID rests solely at the discretion of each user, users must change their default GUID settings for their GUIDs to appear in the access logs. In RealOne Player, the user command for controlling GUID reporting is Tools>Preferences>Connection>Internet Settings.
| For More Information: To review RealNetworks' Consumer Software Privacy Statement, see the Web page located at http://www.realnetworks.com/company/privacy/software. html |
If Windows Media Player and QuickTime Player are configured to send their GUIDs, Helix Universal Proxy records those ID values. If the players do not send GUIDs, Helix Universal Proxy generates an ID for the log. In this case, the same media player may be identified by multiple IDs in the log.
When Helix Universal Proxy can't gather an ID because the client does not
support GUIDs, empty square brackets[ ]appear in the [client_ID] field. If
GUID reporting is disabled on the proxy or media player side, the [client_ID]
field shows a series of zeroes instead of a unique client identifier:
00000000-0000-0000-0000-000000000000 |
The [client_stats_results] field holds connection statistics sent by the client
when it finishes playing a clip, as described in "Client Statistics". If the client
blocks connection statistics, or the statistics cannot be collected, the field
appears as [UNKNOWN].
The following fields hold information about the requested clip:
file_size field lists in bytes the amount of media data in the file. This number is less than the total size of the media file because it does not include the file header and other non-media information. For live broadcasts, file_size is always 0.file_time field gives the total length, in seconds, of media stored in the media file. For live broadcasts, file_time is always 0. For SMIL files, this is always 20.sent_time field expresses in seconds the total playing time of the media transmitted to the client.The resends field lists the number of packets successfully resent because of
transmission errors. The failed_resends field gives the number of packets not
successfully resent in time to correct transmission errors.
The field [stream_components] is recorded only for RealNetworks media
players. It explains the type of material sent, indicated in the following
pattern:
RealAudio_stream RealVideo_stream Event_stream Image_maps |
A value of 1 indicates that the clip includes this type of stream. The value 0
indicates that it does not. Thus, a clip that includes RealVideo and RealAudio
but no event streams or image maps would appear in the access log as this:
1 1 0 0 |
The [start_time] field gives the timestamp of when the clip began to stream,
according to the Helix Universal Proxy clock. It is identical in format to the
timestamp at the beginning of each access record, but does not list the time
difference from Coordinated Universal Time. Here is an example:
[26/Jun/2002:10:05:14] |
The server_address field lists the IP address of the Helix Universal Server or
Helix Universal Proxy that delivered the clip. This may be the origin Helix
Universal Server, a Helix Universal Server which is acting as a receiver, or
another Helix Universal Proxy which is acting as a proxy receiver.
In proxy cache mode, RTSP requests will show the cache's address (usually 127.0.0.1). To find the address of the origin Helix Universal Server, look in the GET field (see GET Statements).
The average_bitrate field lists the average bit rate of the clip in bits per second.
The packets_sent field lists the total number of packets sent to the client.
The presentation_ID field records a number used by all clips in the same SMIL
or Ram presentation. SMIL files are also included in the log, and use the same
number as their clips. For example, if the log entries for a SMIL file, a video
clip, and a GIF image all list presentation ID 437, you can conclude that the
SMIL presentation consisted of that video and image. Helix Universal Proxy
assigns the IDs, which are recorded only with logging style 5, when it
transmits the clips.
The proxy_info field gives information about the type of proxied stream (live or
on-demand) and how Helix Universal Proxy delivered the media stream (pass-
through, pull-split or cache mode.)
One of the following values is recorded:
The GET statement within an access log record shows the path and file name of
each file that Helix Universal Proxy served, as well as the protocol and protocol
version used to stream or broadcast the file. The following sections show
sample entries for GET statements used with different types of on-demand and
live content.
For More Information:
To see the GET statement in context, refer
to "Logging Style".
|
The following table lists the formats in which each type of on-demand content
is shown in the GET statements of the access log. For a SMIL presentation, a
separate record is generated for the SMIL file and for each file in the
presentation. When the logging style is set to 5, you can identify which files
are in the same presentation through the numeric identifier at the end of each
access record.
The following table summarizes the format in which each type of live content is shown in the access log.
All logging styles can include client statistics, which are shown in the
preceding sections as [client_stats_results]. There are four types of statistics,
and the access log can record any combination of them. Each set of statistics is
enclosed in square brackets, and begins with a prefix such as Stat1. If you log
all four types of statistics, for example, the [client_stats_results] field looks like
this:
[Stat1: |
Note that although other access log fields are separated by spaces, there is no space between the closing square bracket of one statistics type and the opening square bracket of the next statistics type. The following example shows logging style 5 (click here) collecting statistics type 1:
207.188.7.125 - - [26/Jun/2002:10:11:03 -0700] "GET real9video.rm RTSP/1.0" |
The following sections describe the information gathered by each of the four statistics types. Statistics 1 and 2 report basic information about playback. Statistics 3 provides information about viewer actions. Statistics 4 reports advanced playback information from RealOne Player. The default logging setting gathers statistics 1 and 2. The following table lists the media players and versions that can send the various statistics types.
Note the following about client statistics:
Stat1 and Stat2 report codec information only about the audio portion of a clip.0 is typically entered for that statistic.[UNKNOWN] is logged.[UNKNOWN] appears in place of that statistics field.Statistics type 1 gathers basic information about the success of media clips received by the client. It also tells what codec the client used to decode the audio portion of the clip. The fields are the following:
[Stat1: received out_of_order missing early late codec] |
These fields provide the following information:
Statistics type 2 provides details about the success of clip delivery, giving information about bandwidth requests. Resent packets are described in detail. This statistics type identifies which transport type was used to make the connection, and which audio codec played the clip. This set of statistics uses the following format:
[Stat2: bandwidth available highest lowest average requested received late |
The fields provide the following information:
Statistics type 3 provides detailed information about viewer action while playing clips, but not while receiving live broadcasts. It addresses advanced streaming features, notably ads and image maps. For example, you can find out when a viewer clicked on an image map or stopped the clip. Because each user may carry out several actions, the access log file may grow rapidly when you collect these statistics. Be sure to review the log file frequently, or set up log file rolling to keep the logs to a manageable size. This statistics type uses the following format:
[Stat3:timestamp|elapsed_time|action|;] |
Records of activity are separated by a semicolon (;). Thus, the Stat3 record of a
viewer pausing, resuming play, and watching to the clip's end looks like the
following:
[Stat3:4360|2107|PAUSE|;8401|2107|RESUME|;12608|6321|STOP|;] |
The initial timestamp field gives the time in milliseconds when the action
occurred. It is relative to the connection time of the client. In the preceding
example, the first timestamp is 4360, meaning the action occurred at 4.360
seconds after the client connected.
The elapsed_time field records how many milliseconds into the clip timeline
that the action occurred. In the preceding example, the PAUSE action occurs at
2107, or 2.107 seconds into the clip timeline. Notice that the RESUME action
also lists the same elapsed time because this action restarts the clip at the
same point where it paused.
The action field records one of several different actions such as STOP or PAUSE,
as described below.
Viewer clicked on the image map. Further information includes:
Resume play after a pause, seek, or stop.
The seek destination point, in milliseconds.
Media player began recording the clip.
Media player stopped recording the clip.
Sent only from RealOne Player, statistics type 4 gathers most of the same information included in statistics type 1 and type 2, adding packet and bandwidth information for each stream. Statistics type 4 reports information for each media stream in the clip, using the following format:
[Stat4:stream_number|mime_type|codec|received|lost|resent|average_bandwidth |
The following is an example type 4 log entry for a RealVideo Clip:
[Stat4:2 audio/x-pn-realaudio|44_kbps_Stereo_Music_High_Response_-_RA8 |
| Note: If you turn on statistics type 4 as well as statistics type 1 or 2, RealOne Player reports only statistics type 4. Other media players, however, will report statistics type 1 or 2, but not statistics type 4. |
The stream_number field indicates how many media streams the clip contains.
A video clip might have two streams, for example, one for the audio track and
one for the visual track. Following this, information for each stream is
reported.
Helix Universal Proxy reports information for each stream. Information ends with a semicolon. For each stream, the following fields are reported:
The transport field indicates the transport protocol used for the connection.
Values are:
0 |
IP Multicast |
1 |
UDP |
2 |
TCP |
3 |
HTTP cloaked |
Three turboplay fields indicate the use and results of the RealOne Player
TurboPlay feature. The three fields are separated by pipes, as shown here:
1|513234|1120 |
The following table lists the possible field values. Values for the second and third field vary depending on whether TurboPlay is on or off, as indicated in the first field.
The duration field gives the time in milliseconds between the initial client
request and the first data packet received by the client.
The clip_end field lists the reason the presentation ended. Possible values are:
0 |
end of presentation reached |
1 |
stop command issued |
2 |
reconnection required |
3 |
redirection |
PNR_n |
error code n occurred |
If you are also managing a Helix Universal Server, you know that it records similar information about client requests in the server access log as Helix Universal Proxy records in the proxy access log. Additionally, the server access log records extra information (like proxy IP address) when a clip is delivered by either proxy pull splitting or proxy cache.
Helix Universal Server and Helix Universal Proxy access log settings are independent of each other; each installation has independent log files. For example, as someone managing Helix Universal Proxy, you can configure it to record information with Logging Style 0. You can then configure Helix Universal Server to collect all the information of Logging Style 5, resulting in logs that gather different amounts of information.
The following sections explain how to modify the logging of access and error records. Logging is turned on by default. You may want to change certain default options, however.
The access log is preconfigured to gather basic client statistics for media player requests. You may want to change the logging style and client statistics types, as well as set up log file rolling.
| To modify access logging: |
5. For information about the logging style, see "Logging Style".Yes from the Disable Client GUIDs pull-down list. This eliminates the collection of client global identifiers. For more information, see "Client Identifier".30, for example, means that the client sends statistics, and Helix Universal Proxy creates new log entry, every 30 seconds. If you set this to 0 (zero), the client sends statistics once when the presentation ends.Type 1 and Type 2. For more information, see "Client Statistics".| Tip: If you gather statistics type 3 or 4, the access log file size will grow rapidly. In this case, be sure to review the log file frequently, or use log file rolling. |
| Tip: Generally, you limit log files by frequency or size. You can select both methods, however, to create log files according to the first limit reached. For example, you can create a new log file whenever the preceding file reaches 10 Megabytes in size, or has recorded 3 days of activity, whichever comes first. |
Logs subdirectory of the Helix Universal Proxy main directory. The default file name of the access log file is proxy.log.| Note: If Proxy Log File is blank, Helix Universal Proxy records access information in a file named, proxy.log, located in the same directory as the Helix Universal Proxy executable file. |
Using the error log requires no configuration. You may want to set up log file rolling, though, or specify a different location and name for the error log. For basic information on error log syntax, see "Error Log".
| To modify the error log: |
| Tip: Generally, you limit log files by frequency or size. You can select both methods, however, to create log files according to the first limit reached. For example, you can create a new log file whenever the preceding file reaches 10 Megabytes in size, or has recorded 3 days of activity, whichever comes first. |
Logs subdirectory of the Helix Universal Proxy main directory. The default name of the error log file is proxyerr.log.|
|
© 2002 RealNetworks, Inc. All rights reserved.
For more information, visit RealNetworks Click here if the Table of Contents frame is not visible at the left side of your screen. |