Operations - Activity Logs

The Operations screen, accessible within an app's Development and Production environments, provides detailed information about job processing, both ad-hoc and recurring, as well as the ability to modify job settings. One of the tabs on the Operations screen is 'Activity Logs', which displays various information and logs of the historical executions of jobs. In this article, we will cover the following topics related to Activity Logs:

Understanding Job Cards

When you navigate to the Operation screens of any environment, the left side displays one job card for each job that has been created for the app within the current environment, be it development or production. Each job card provides the following information:

  • Status of the last execution
  • Next schedule (only available in production)
  • Job name
  • Created - the creation date of the job
  • Last Run - the end time of the last execution
  • Duration - the duration of the last execution

Activity Logs Grid

Within the Operations screen, clicking on an individual job card will display details of that job.  The Activity Logs tab shows various information and logs of the historical executions of the job.

The grid within Activity Logs shows one row per each instance of the job's execution.  The most recent execution is shown at the top, which is typically the most important instance an administrator is concerned with, but the historical executions are also seen and can be reviewed as needed. 

The grid shows the following information:

  • Execution - The execution ID is a unique identifier for each job execution. IDs are unique within a single job but are reused across different jobs. Each rerun of a job generates a new execution ID.
  • Status - The Status column displays the current status of the job. The far left column of the grid and the job card are color-coded to reflect each status:
    • Starting (gray) - The job is in its initialization stages.
    • Running (blue) - The job is actively running.
    • Aborted (yellow) - Job has been aborted (manual intervention required to perform).
    • Completed (green) - The job has been completed successfully.
    • Failed (red) - The job has been completed with an error.
  • Start Time - The Start Time column displays the start date and time of the job.  The date format will be shown in the language of the browser.
  • End Time - The End Time column displays the end date and time of the job.  The date format will be shown in the language of the browser.
  • Duration (hh:mm:ss) - The Duration column displays the difference between Start Time and End Time if the job is concluded, i.e. Completed, Failed, or Aborted.  If the job is in the status Starting or Running then the Duration column displays the difference between Start Time and the user's computer current time.
  • Time Span - The Time Span column indicates the From Date and To Date parameter values the job was run with.
  • Type - The Type column indicates the method in which the job was started, i.e. Manual or Scheduled (only available in production).
  • Submitted By - The Submitted By column indicates the user that either submitted the job manually or last modified the job that ran on a scheduled basis.

Activity Log Grid Actions

  • Sorting:
    • All columns can be sorted by clicking on the column header.
    • Clicking once sorts the column in ascending order, indicated by an up arrow (↑).
    • Clicking again sorts the column in descending order, indicated by a down arrow (↓).
    • Clicking a third time removes sorting on the column, indicated by no arrow.
  • Filtering: 
    • All columns can be filtered by clicking on the column filter icon (mceclip2.png)
    • Date columns (Start Time and End Time) can be filtered directly using the calendar icon (mceclip3.png).
  • Viewing Logs:
    • View Logs: Job-level logs can be accessed by clicking the pop-up icon (mceclip4.png) within the status column. More detailed logs are available at the individual step level within a job(see below. You can also click on 'View Detailed Logs' to see logs at a very detailed level for the complete Job. (Refer Screenshot below)
    • Copy Logs: Within the Execution Logs window, you can click the copy icon (mceclip5.png) to copy the logs to your clipboard.

 

Job Execution Insights (Step by Step)

Selecting a job in the Activity Logs grid opens the Execution Details panel on the right. This panel provides a detailed breakdown of the chosen job, including:

  • Job Summary: Key information about the job, such as name, status, and creation date, is readily available, mirroring what you see on the grid.
  • Step-by-step breakdown: This view is particularly helpful for users in the following ways:
    • Troubleshooting Job Failures: When a job fails, detailed logs from the failing step can pinpoint the exact issue. This allows you to diagnose problems quickly and efficiently, saving time and effort.
    • Performance Monitoring: By analyzing execution times for each step, you can identify bottlenecks that might be slowing down your jobs. This helps you optimize job performance and ensure smooth operation.
    • Verification and Auditing: Step-level logs provide a record of each step's execution. You can use these logs to verify that successful steps were completed as expected and identify any errors that might have occurred during specific parts of the job.
    • Root Cause Analysis: When investigating job issues, step-by-step logs help you trace the root cause of problems. This allows you to fix issues effectively and prevent them from recurring in the future.

Let's understand the execution details panel step by step:

Top Panel

  • The top portion of the panel displays job-level information, mirroring the details seen on the grid. If the job is currently running, an abort icon( mceclip5.png)will appear next to the Duration field. Clicking this icon allows you to abort the running job.
  • By default, all job steps are shown in a collapsed state. You can expand all steps simultaneously by clicking 'Expand All,' or you can expand individual steps as needed, as demonstrated in the above screenshot for the second step (AA Loader).

Bottom Panel

  • The bottom panel of the Execution Details panel displays a breakdown of each step involved in running the job. This includes:
    • All configured steps: Every step defined in the "Configure" tab of the Operations screen is listed here, regardless of whether it ran successfully or not.
    • Cluster management steps: Even steps that manage the cluster environment, like "Inframanager_create_cluster" (startup) and "Inframanager_terminate_cluster" (shutdown), are shown though they aren't part of the configured job steps themselves.

Let's understand each field shown for individual step:

  • Name - The name of the individual step is shown in bold.
  • Duration (hh:mm:ss) - The duration of the individual step is shown in the top right of the section.
  • Start Time - The Start Time field displays the start date and time of the job step.  The date format will be shown in the language of the browser.  If the job step has not yet begun or the job was aborted and failed and thus a configured job step did not begin then the Start Time field will display N/A.
  • End Time - The End Time field displays the end date and time of the job step.  The date format will be shown in the language of the browser.  If the job step is running or did not run because the job was aborted or a prior job step failed then the End Time field will display N/A.
  • Process Mode - The Process Mode field displays the value of the setting at the time of the job step execution.  The process mode setting provides the app with instructions on how to handle the processing of incoming data, such as dropping any data or only processing new data.
  • Runtime Type - Seen by expanding the details of the job step, the Runtime Type field displays the type of the runtime that this step was run on.
  • Runtime - Seen by expanding the details of the job step, the Runtime field displays the individual runtime that this step was run on.
  • Status Icon - The following icons are shown representing the status of each step:
    • Pending - The job has started, but the job step is pending; a blue pie chart icon is shown (mceclip3.png).
    • Running - The job step is actively running; a blue rotating icon is shown (mceclip4.png).
    • Aborted - The job step has been aborted (manual intervention required to perform); a yellow pause icon is shown (mceclip2.png).
    • Not Started - The job has been aborted before attempting to run the configured step; a blank circle icon is shown (mceclip0.png).
    • Completed - The job step has been completed successfully; a green check icon is shown (mceclip0.png).
    • Failed - The job step has been completed with an error; a red exclamation mark is shown (mceclip1.png).
  • View Logs - The logs at the step level can be seen by clicking the pop-up icon (mceclip4.png) seen to the right of the duration.  Within the Execution Logs window, the copy icon (mceclip5.png) can be clicked to copy the logs to the user's clipboard.

In essence, the Execution Details panel with its step-level breakdown equips you with the information you need to manage your jobs efficiently. You can quickly troubleshoot issues, optimize performance, and ensure the smooth operation of your application.