6. Events and logging

This section describes how to monitor and configure logging using the CLI, REST, IPMI and Redfish interfaces.

6.1. BMC command line

The standard Linux journal is available with the journalctl command on the BMC.

6.2. REST API

You can perform logging operations through the REST API either by sending curl queries to the URI or by using openbmctool.py utility. Table 6.1 describes the commands available.

Table 6.1 Logging operations using the REST interface
Command	Description
`list`	List log entries (all types), log managers and log configurations available on the system.
List all logs: $ curl -k https://<bmcip>/xyz/openbmc_project/logging/list -u <bmcuser>:<bmcpass>
`enumerate`	Show detailed information about log entries (all types), log managers and log configurations available on the system.
Enumerate all logs: $ curl -k https://<bmcip>/xyz/openbmc_project/logging/enumerate -u <bmcuser>:<bmcpass> Enumerate log with specific ID: $ curl -k https://<bmcip>/xyz/openbmc_project/logging/entry/<logid>/enumerate -u <bmcuser>:<bmcpass>
`resolve`	Clear a SEL
$ curl -k -H "Content-Type: application/json" -d '{"data":true}' -X PUT https://<bmcip>/xyz/openbmc_project/logging/entry/<eventId>/attr/Resolved -u <bmcuser>:<bmcpass>
Configuration	Configure syslog logging on the BMC.
Configure syslog server address and port: $ curl -k -H "Content-Type: application/json" -X PUT -d '{"data":<port>}' https://<bmcuser>:<bmcpass>@<bmcip>/xyz/openbmc_project/logging/config/remote/attr/Port $ curl -k -H "Content-Type: application/json" -X PUT -d '{"data":"<address>"}' https://<bmcuser>:<bmcpass>@<bmcip>/xyz/openbmc_project/logging/config/remote/attr/Address $ python3 openbmctool.py -H <bmcip> -U <bmcuser> -P <bmcpass> logging remote_logging_config -a ADDRESS -p PORT View syslog configuration: $ python3 openbmctool.py -H <bmcip> -U <bmcuser> -P <bmcpass> logging remote_logging view Disable syslog: $ python3 openbmctool.py -H <bmcip> -U <bmcuser> -P <bmcpass> logging remote_logging disable

Example of listing log entries, log managers and logging configuration:

$ curl -k https://<bmcip>/xyz/openbmc_project/logging/list -u <bmcuser>:<bmcpass>
{
  "data": [
    "/xyz/openbmc_project/logging/config",
    "/xyz/openbmc_project/logging/config/remote",
    "/xyz/openbmc_project/logging/entry",
    "/xyz/openbmc_project/logging/entry/15",
    "/xyz/openbmc_project/logging/entry/16",
    "/xyz/openbmc_project/logging/entry/17",
    "/xyz/openbmc_project/logging/entry/18",
    "/xyz/openbmc_project/logging/entry/19",
    "/xyz/openbmc_project/logging/entry/20",
    "/xyz/openbmc_project/logging/entry/21",
    "/xyz/openbmc_project/logging/entry/21/callout",
    "/xyz/openbmc_project/logging/internal",
    "/xyz/openbmc_project/logging/internal/manager",
    "/xyz/openbmc_project/logging/rest_api_logs"
  ],
  "message": "200 OK",
  "status": "ok"
}

You can get more details about the available log entries with the enumerate command.

An example of a sensor threshold crossing event is shown as below:

$ curl -k https://<bmcip>/xyz/openbmc_project/logging/entry/15/enumerate -u <bmcuser>:<bmcpass>
{
  "data": {
    "/xyz/openbmc_project/logging/entry/15": {
      "AdditionalData": [
        "EVENT_DIR=1",
        "GENERATOR_ID=32",
        "RECORD_TYPE=2",
        "SENSOR_DATA=57F0CA",
        "SENSOR_PATH=/xyz/openbmc_project/sensors/voltage/main_12v0_volt",
        "_PID=414"
      ],
      "Id": 15,
      "Message": "xyz.openbmc_project.Logging.SEL.Error.Created",
      "Purpose": "xyz.openbmc_project.Software.Version.VersionPurpose.BMC",
      "Resolved": false,
      "Severity": "xyz.openbmc_project.Logging.Entry.Level.Informational",
      "Timestamp": 1641218828241,
      "Version": "2.7.0-1918-ge2fed180b",
      "associations": []
    }
  },
  "message": "200 OK",
  "status": "ok"
}

Major information about the event corresponding to the given log entry is available under the AdditionalData key. When the AdditionalData key includes the RECORD_TYPE=2 field, data is encoded into a format expected by IPMI, and can be decoded as described in the IPMI section below.

6.3. IPMI

You can read all the events for the IPU-Machine using the IPMI commands shown in Table 6.2.

Table 6.2 SEL operations using IPMI interface
Command	Description
SEL clear	Clear all event logs
ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel clear
SEL list	Display list of events
ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel list
SEL elist	Display list of events with extended information
ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist

Below are examples of SEL entries seen when sensors are crossing threshold, or SEL entries seen when inventory items are in error(such as not present or not functional).

Note

Sensor threshold values can be changed using sensor thresh command described in Table 2.5 Sensor monitoring. The examples below are assuming default sensor thresholds.

6.3.1. Sensors crossing threshold SEL logs examples

When the inlet sensor goes above 45°C or the exhaust temperature sensor goes above 75°C, threshold violation asserted events are logged within the system.

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
100 | 03/01/2022 | 15:07:13 | Temperature inlet_temp | Upper Critical going high | Asserted | Reading 45.57 > Threshold 45.00 degrees C
101 | 03/01/2022 | 15:07:19 | Temperature exhaust_temp | Upper Critical going high | Asserted | Reading 76.20 > Threshold 75.00 degrees C

When the above sensors goes back to the normal working range, threshold violation deasserted events are logged within the system.

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
102 | 03/01/2022 | 15:09:15 | Temperature inlet_temp | Upper Critical going high | Deasserted | Reading 43.00 > Threshold 45.00 degrees C
103 | 03/01/2022 | 15:09:20 | Temperature exhaust_temp | Upper Critical going high | Deasserted | Reading 74.00 > Threshold 75.00 degrees C

In addition, if you have configured an SNMP manager (see SNMP trap) for receiving SNMP event traps, you can see the following traps associated with the SEL entries. You can use these to identify the faulty sensor (in this case, the inlet sensor going over 45°C).

2020-05-11 08:06:24 <bmc_fqdomain> [UDP: [<bmcip>]:<port>->[<snmpmgrip>]:<dport>]:
iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.49871.1.0.0.1      iso.3.6.1.4.1.49871.1.0.1.1 = Gauge32: 47   iso.3.6.1.4.1.49871.1.0.1.2 = Opaque: UInt64: 168713071370436978 iso.3.6.1.4.1.49871.1.0.1.3 = INTEGER: 3    iso.3.6.1.4.1.49871.1.0.1.4 = STRING: "xyz.openbmc_project.Sensor.Threshold.Error.CriticalHigh - SENSOR_DATA=|/xyz/openbmc_project/sensors/temperature/inlet:Value=45000|"

2020-05-11 08:06:24 <bmc_fqdomain> [UDP: [<bmcip>]:60408->[<snmpmgrip>]:<dport>]:
iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.49871.1.0.0.1      iso.3.6.1.4.1.49871.1.0.1.1 = Gauge32: 48   iso.3.6.1.4.1.49871.1.0.1.2 = Opaque: UInt64: 168728099461005682 iso.3.6.1.4.1.49871.1.0.1.3 = INTEGER: 3    iso.3.6.1.4.1.49871.1.0.1.4 = STRING: "xyz.openbmc_project.State.Shutdown.ThermalEvent.Error.Ambient - _PID=265"

An overview of all sensors that are capable of creating SEL entries is available in Table 4.6.

6.3.2. Inventory Items failure or present detection error SEL logs examples

When a power supply failure or removal is detected, you will see the following SEL entry:

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
XX | 05/11/2020 | 08:39:53 AM CEST | Power Supply #0x0a | Presence detected | Asserted

You need to cross-check with the SDR to identify if this is a failure or absence of the power supply.

In the case of a functional error, you will see an entry similar to the following entry in SEL logs:
```
powersupply1     | 0Ah | ok  | 10.2 | Presence Detected, Failure detected
```
In the case of a presence error, you will see an entry similar to the following entry in SEL logs:
```
powersupply1     | 0Ah | ok  | 10.2 |
```
In case of AC lost, you will see an entry similar to the following entry in SEL logs:
```
Power Supply powersupply0 | Power Supply AC lost | Asserted
```
When AC is back, you will see an entry similar to the following entry in SEL logs:
```
Power Supply powersupply0 | Power Supply AC lost | Deasserted
```

For this power supply SEL entry, the following trap is received on the SNMP manager.

2020-05-12 13:38:14 <bmc_fqdomain> [UDP: [<bmcip>]:<sport>->[<snmpgrip>]:<dport>]:
iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.49871.1.0.0.1      iso.3.6.1.4.1.49871.1.0.1.1 = Gauge32: 52       iso.3.6.1.4.1.49871.1.0.1.2 = Opaque: UInt64: 625323637452308850       iso.3.6.1.4.1.49871.1.0.1.3 = INTEGER: 3        iso.3.6.1.4.1.49871.1.0.1.4 = STRING: "xyz.openbmc_project.Inventory.Error.Nonfunctional - CALLOUT_INVENTORY_PATH=/xyz/openbmc_project/inventory/system/chassis/powersupply1"

When an IPU failure is detected, you will see the following SEL entry:

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
XX | 05/12/2020 | 04:13:22 PM CEST | Processor #0x11 | Disabled | Asserted

You need to cross-check with the SDR to identify if it is a functional or presence error.

In the case of a functional error, you will see the following output:
```
ipu0             | 11h | ok  | 45.1 | Presence detected, Disabled
```
In the case of a presence error, you will see the following output:
```
ipu0             | 11h | ok  | 45.1 |
```

Note

RNICs are also defined as processor/IO modules and similar SEL/SDR entries will appear in the case of an RNIC failure.

For the above IPU SEL entry, a trap similar to that shown below will be received by the configured SNMP manager.

2020-05-12 22:53:45 ipum.example.com [UDP: [<bmcip>]:<sport>->[<snmpmgrip>]:<dport>]:
iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.49871.1.0.0.1      iso.3.6.1.4.1.49871.1.0.1.1 = Gauge32: 63    iso.3.6.1.4.1.49871.1.0.1.2 = Opaque: UInt64: 768481056411091314        iso.3.6.1.4.1.49871.1.0.1.3 = INTEGER: 3     iso.3.6.1.4.1.49871.1.0.1.4 = STRING: "xyz.openbmc_project.Inventory.Error.Nonfunctional - CALLOUT_INVENTORY_PATH=/xyz/openbmc_project/inventory/system/chassis/motherboard/ipu0"

When an NVMe failure is detected, an SEL entry like the following will be logged:

XX | 05/13/2020 | 10:18:37 AM CEST | Drive Slot / Bay #0x42 | Drive Fault | Asserted

A cross check with SDR is needed to figure out if it is a functional or presence error.

In the case of a functional error:

nvme0            | 42h | ok  |  4.1 | Drive Present, Drive Fault

In the case of a presence error:
```
nvme0            | 42h | ok  |  4.1 |
```

For the above NVMe SEL entry, a trap like the following is received on the SNMP manager:

2020-05-13 10:18:37 ipum.example.com [UDP: [<bmcip>]:<sport>->[snmpmgrip]:<dport>]:
iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.49871.1.0.0.1      iso.3.6.1.4.1.49871.1.0.1.1 = Gauge32: 71       iso.3.6.1.4.1.49871.1.0.1.2 = Opaque: UInt64: 944969603430220146        iso.3.6.1.4.1.49871.1.0.1.3 = INTEGER: 3        iso.3.6.1.4.1.49871.1.0.1.4 = STRING: "xyz.openbmc_project.Inventory.Error.Nonfunctional - CALLOUT_INVENTORY_PATH=/xyz/openbmc_project/inventory/system/chassis/motherboard/nvme0"

When an event is de-asserted, a generic system entry with Deasserted state is recorded in the SEL as shown below:
```
05/13/2020 | 08:24:21 AM CEST | System Event #0x90 | Undetermined system hardware failure | Deasserted
```

6.4. Redfish

You can do logging operations through Redfish interface either by sending curl queries or browsing the URI.

You can find SEL endpoints in the Systems collection available at https://<bmcip>/redfish/v1/Systems/system/LogServices/.

You can find journal endpoints in the Manager collection available at https://<bmcip>/redfish/v1/Managers/bmc/LogServices.

Table 6.3 Logging operations using Redfish interface
Command	Description
List SEL	List SEL entries.
$ curl -k https://<bmcip>/redfish/v1/Systems/system/LogServices/EventLog/Entries -u <bmcuser>:<bmcpass>
Delete logging entries	Delete system event entries.
$ curl -k https://<bmcip>/redfish/v1/Systems/system/LogServices/EventLog/Actions/LogService.ClearLog -u <bmcuser>:<bmcpass>
List journal logs	List systemd journal logs through Redfish.
$ curl -k https://<bmcip>/redfish/v1/Managers/bmc/LogServices/Journal/Entries -u <bmcuser>:<bmcpass>

Note

Redfish SELs do not include the inventory item that created the log.

Example output from the list SEL command is shown below:

$ curl -k https://<bmcip>/redfish/v1/Systems/system/LogServices/EventLog/Entries -u <bmcuser>:<bmcpass>
{
      "@odata.context": "/redfish/v1/$metadata#LogEntryCollection.LogEntryCollection",
      "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries",
      "@odata.type": "#LogEntryCollection.LogEntryCollection",
      "Description": "Collection of System Event Log Entries",
      "Members": [
        {
          "@odata.context": "/redfish/v1/$metadata#LogEntry.LogEntry",
          "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries/15",
          "@odata.type": "#LogEntry.v1_4_0.LogEntry",
          "Created": "2020-05-11T11:45:21+00:00",
          "EntryType": "Event",
          "Id": "15",
          "Message": "xyz.openbmc_project.Inventory.Error.Nonfunctional",
          "Name": "System Event Log Entry",
          "Severity": "Critical"
        },
        {
          "@odata.context": "/redfish/v1/$metadata#LogEntry.LogEntry",
          "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries/16",
          "@odata.type": "#LogEntry.v1_4_0.LogEntry",
          "Created": "2020-05-11T11:45:23+00:00",
          "EntryType": "Event",
          "Id": "16",
          "Message": "xyz.openbmc_project.Inventory.Error.Nonfunctional",
          "Name": "System Event Log Entry",
          "Severity": "Critical"
        },
      ],
      "[email protected]": 2,
      "Name": "System Event Log Entries"
}

Example output from the command to list journal entries is shown below:

$ curl -k https://<bmcip>/redfish/v1/Managers/bmc/LogServices/Journal/Entries -u <bmcuser>:<bmcpass>
{
   "@odata.context": "/redfish/v1/$metadata#LogEntryCollection.LogEntryCollection",
   "@odata.id": "/redfish/v1/Managers/bmc/LogServices/BmcLog/Entries",
   "@odata.type": "#LogEntryCollection.LogEntryCollection",
   "Description": "Collection of BMC Journal Entries",
   "Members": [
     {
       "@odata.context": "/redfish/v1/$metadata#LogEntry.LogEntry",
       "@odata.id": "/redfish/v1/Managers/bmc/LogServices/Journal/Entries/1589216068566513",
       "@odata.type": "#LogEntry.v1_4_0.LogEntry",
       "Created": "2020-05-11T16:54:28+00:00",
       "EntryType": "Oem",
       "Id": "1589216068566513",
       "Message": "Booting Linux on physical CPU 0x0",
       "Name": "BMC Journal Entry",
       "OemRecordFormat": "BMC Journal Entry",
       "Severity": "OK"
     },
     ....
   ],
   "[email protected]": 2096,
   "[email protected]": "/redfish/v1/Managers/bmc/LogServices/Journal/Entries?$skip=1000",
   "Name": "Open BMC Journal Entries"
 }

6.5. List of supported events

6.5.1. Sensor crossing threshold events

The following threshold events are supported for threshold/analog sensors over IPMI.

Threshold Events | Description

Upper Critical Going High Assert (ucr+)

Sensor value has crossed critical high threshold (Upper critical going high | Asserted)

Upper Critical Going High Deassert (ucr+)

Sensor value has gone back below critical high threshold (Upper critical going high | Desserted)

Upper Non-Critical Going High Assert (unc+)

Sensor value has crossed warning high threshold (Upper Non-critical going high | Asserted)

Upper Non-Critical Going High Deassert (unc+)

Sensor value has gone back below warning high threshold (Upper Non-critical going high | Desserted)

Lower Critical Going Low Assert (lcr-)

Sensor value has crossed critical low threshold (Lower critical going low | Asserted)

Lower Critical Going Low Deassert (lcr-)

Sensor value has gone back above critical low threshold (Lower critical going low | Desserted)

Lower Non-Critical Going Low Assert (lnc-)

Sensor value has crossed warning low threshold (Lower Non-critical going low | Asserted)

Lower Non-Critical Going Low Deassert(lnc-)

Sensor value has gone back above warning low threshold (Lower Non-critical going low | Desserted)

An example of sensor crossing threshold event is below:

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
1a | 06/20/2021 | 07:41:04 | Voltage main_12v0_volt | Upper Non-critical going high | Asserted | Reading 13.20 > Threshold 11.80 Volts
1b | 06/20/2021 | 07:44:04 | Voltage main_12v0_volt | Upper Non-critical going high | Deasserted | Reading 11.10 > Threshold 11.80 Volts

By default all above threshold events are enabled for all analog sensors. In order to get the disable/enable status of assert/deassert events for a threshold sensor, you can check the “Assertions Enabled / Deassertion Enabled” status of the sensor using sensor get command. In example below all the above events are enabled.

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sensor get main_12v0_volt

  Locating sensor record...
  Sensor ID              : main_12v0_volt (0x7a)
  Entity ID             : 7.11
  Sensor Type (Threshold)  : Voltage
  Sensor Reading        : 11.988 (+/- 0) Volts
  Status                : ok
  Lower Non-Recoverable : na
  Lower Critical        : 10.800
  Lower Non-Critical    : 11.016
  Upper Non-Critical    : 13.014
  Upper Critical        : 13.176
  Upper Non-Recoverable : na
  Positive Hysteresis   : Unspecified
  Negative Hysteresis   : Unspecified
  Assertions Enabled    : lnc- lcr- unc+ ucr+
  Deassertions Enabled  : lnc- lcr- unc+ ucr+

You can use ipmi raw command 0x28, netfn 0x04 do disable individual events or all events on the desired sensor. Example below is showing how to disable all events for the above sensor.

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> raw 0x04 0x28 0x7a 0x40

After this double checking the sensor status shows that event have been disabled on the sensor:

$ ipmitool -I lanplus -C 3 -p 623 -U <bmcuser> -P <bmcpass> -H <bmcip> sensor get main_12v0_volt

Locating sensor record...
Sensor ID              : main_12v0_volt (0x7a)
Entity ID             : 7.11
Sensor Type (Threshold)  : Voltage
Sensor Reading        : 11.988 (+/- 0) Volts
Status                : ok
Lower Non-Recoverable : na
Lower Critical        : 10.800
Lower Non-Critical    : 11.016
Upper Non-Critical    : 13.014
Upper Critical        : 13.176
Upper Non-Recoverable : na
Positive Hysteresis   : Unspecified
Negative Hysteresis   : Unspecified
Event Enable          : Event Messages Disabled
Assertions Enabled    :

Note

Please see Section 4, Sensor monitoring for the list of monitored sensors and event/actions associated with them.

You can also use the Redfish or REST interfaces for a more detailed description of sensor related events, including the sensor name and the value of the sensor when the threshold was crossed.

$ curl -k https://<bmcip>/redfish/v1/Systems/system/LogServices/EventLog/Entries -u <bmcuser>:<bmcpass>
{
  "@odata.context": "/redfish/v1/$metadata#LogEntryCollection.LogEntryCollection",
  "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries",
  "@odata.type": "#LogEntryCollection.LogEntryCollection",
  "Description": "Collection of System Event Log Entries",
  "Members": [
    {
      "@odata.context": "/redfish/v1/$metadata#LogEntry.LogEntry",
      "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries/26",
      "@odata.type": "#LogEntry.v1_4_0.LogEntry",
      "Created": "2021-06-20T07:41:04+00:00",
      "EntryType": "Event",
      "Id": "26",
      "Message": "xyz.openbmc_project.Sensor.Threshold.Error.CriticalHigh",
      "Name": "System Event Log Entry",
      "Severity": "Critical"
    },
    {
      "@odata.context": "/redfish/v1/$metadata#LogEntry.LogEntry",
      "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries/27",
      "@odata.type": "#LogEntry.v1_4_0.LogEntry",
      "Created": "2021-06-20T07:41:06+00:00",
      "EntryType": "Event",
      "Id": "27",
      "Message": "xyz.openbmc_project.State.Shutdown.ThermalEvent.Error.Ambient",
      "Name": "System Event Log Entry",
      "Severity": "Critical"
    }
  ],
  "[email protected]": 2,
  "Name": "System Event Log Entries"
}

Note

REST and Redfish interfaces represent sensor crossing threshold events with “xyz.openbmc_project.Sensor.Threshold.Error.CriticalHigh” or “xyz.openbmc_project.Sensor.Threshold.Error.WarningHigh” events with the sensor name as metadata. If the event is a shutdown event, it gets logged with “xyz.openbmc_project.State.Shutdown.ThermalEvent.Error.Ambient” message in the event logs.

6.5.2. Inventory items presence or functional-state events

Presence and functional state of the following inventory items are tracked.

PSU0 and PSU1

FAN0 to FAN4

IPU0 to IPU3

RNIC0 and RNIC1

NVMe0

Note

For the inventory items listed under number 3 to 5, the functional state is determined by checking the state of PCIe link of the endpoint to be up as Gen4x8 links.

Note

The fans are numbered 1 to 5 in the hardware, but 0 to 4 in the software. The fan numbered 1 in the hardware corresponds to fan0_0 and fan0_1 in the software. See Section 16, Fan PWM-to-speed mapping for more information.

The following are examples of SEL logs when power supply presence or functional state changes.

# Powersupply0 not present
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
   1c | 06/20/2021 | 13:22:58 | Power Supply #0x09 | Presence detected | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep powersupply
   powersupply0     | 09h | ok  | 10.1 |
   powersupply1     | 0Ah | ok  | 10.2 |  Presence detected

# Powersupply1 not present
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
   1d | 06/20/2021 | 13:41:52 | Power Supply #0x0a | Presence detected | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep powersupply
   powersupply0     | 09h | ok  | 10.1 |  Presence detected
   powersupply1     | 0Ah | ok  | 10.2 |

# Powersupply0 back to present or functional
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
   1c | 06/20/2021 | 13:22:58 | System Event #0x90 | Undetermined system hardware failure | Deasserted
# Powersupply1 back to present
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
   1d | 06/20/2021 | 13:22:58 | System Event #0x90 | Undetermined system hardware failure | Deasserted

# Powersupply0 not functional
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
   20 | 06/20/2021 | 14:40:41 | Power Supply #0x09 | Presence detected | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep powersupply
  powersupply0     | 09h | ok  | 10.1 | Presence detected, Failure detected
  powersupply1     | 0Ah | ok  | 10.2 | Presence detected

# Powersupply1 not functional
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
   21 | 06/20/2021 | 14:40:41 | Power Supply #0x09 | Presence detected | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep powersupply
  powersupply0     | 09h | ok  | 10.1 | Presence detected
  powersupply1     | 0Ah | ok  | 10.2 | Presence detected, Failure detected

The following are examples of SEL logs when fan module presence or functional state changes.

# Fan module 0 not present
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
  22 | 06/20/2021 | 14:49:41 | Fan #0x3c | State Deasserted | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep fan
  fan0             | 3Ch | ok  | 29.11 |
  fan1             | 3Dh | ok  | 29.12 | State Deasserted
  fan2             | 3Eh | ok  | 29.13 | State Deasserted
  fan3             | 3Fh | ok  | 29.14 | State Deasserted
  fan4             | 40h | ok  | 29.15 | State Deasserted

# Fan module 0 not functional
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
  22 | 06/20/2021 | 14:49:41 | Fan #0x3c | State Deasserted | Asserted

# Fan module present or functional state back to ok
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
  22 | 06/20/2021 | 14:59:08 | System Event #0x90 | Undetermined system hardware failure | Deasserted

The following are examples of SEL logs when the “ipu0” inventory item is not detected as a PCIe endpoint, or their functional state changes.

# IPU 0 is not detected as present
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
  23 | 06/20/2021 | 15:06:03 | Processor #0x11 | Disabled | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep ipu
  ipu0             | 11h | ok  | 45.1 |
  ipu1             | 12h | ok  | 45.2 | Presence detected
  ipu2             | 13h | ok  | 45.3 | Presence detected
  ipu3             | 14h | ok  | 45.4 | Presence detected

# IPU 0 is not detected as functional
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
24 | 06/20/2021 | 15:12:36 | Processor #0x11 | Disabled | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep ipu
  ipu0             | 11h | ok  | 45.1 | Presence detected, Disabled
  ipu1             | 12h | ok  | 45.2 | Presence detected
  ipu2             | 13h | ok  | 45.3 | Presence detected
  ipu3             | 14h | ok  | 45.4 | Presence detected

The following are examples of SEL logs when the “rnic0” inventory item is not detected as a PCIe endpoint, or its functional state changes.

# RNIC 0 is not detected as prenset
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
25 | 06/20/2021 | 16:01:01 | Processor #0x17 | Disabled | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep rnic
rnic0            | 17h | ok  | 45.7 |
rnic1            | 18h | ok  | 45.8 | Presence detected

# RNIC 0 is not detected as functional
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
26 | 06/20/2021 | 16:05:13 | Processor #0x17 | Disabled | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep rnic
rnic0            | 17h | ok  | 45.7 | Presence detected, Disabled
rnic1            | 18h | ok  | 45.8 | Presence detected

The following are examples of SEL logs when the “nvme0” inventory item is not detected as a PCIe endpoint, or its functional state changes.

# NVMe drive is detected as not present
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
27 | 06/20/2021 | 16:07:57 | Drive Slot / Bay #0x42 | Drive Fault | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep nvme
nvme0            | 42h | ok  |  4.1 |

# NVMe drive is detected as not functional
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
28 | 06/20/2021 | 16:10:27 | Drive Slot / Bay #0x42 | Drive Fault | Asserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep nvme
nvme0            | 42h | ok  |  4.1 | Drive Present, Drive Fault

# When presence-state error or functional-state error is resolved.
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sel elist
27 | 06/20/2021 | 16:07:57 | System Event #0x90 | Undetermined system hardware failure | Deasserted
$ ipmitool -I lanplus  -U <bmcuser> -P <bmcpass> -H <bmcip> sdr elist | grep nvme
nvme0            | 42h | ok  |  4.1 | Drive Present

Note

REST and Redfish represent inventory presence or functional-error events with “xyz.openbmc_project.Inventory.Error.NonPresent” or “xyz.openbmc_project.Inventory.Error.Nonfunctional” with the name of the inventory item name that is in a faulty state as the metadata in the event logs.

Search help

6. Events and logging

6.1. BMC command line

6.2. REST API

6.3. IPMI

6.3.1. Sensors crossing threshold SEL logs examples

6.3.2. Inventory Items failure or present detection error SEL logs examples

6.4. Redfish

6.5. List of supported events

6.5.1. Sensor crossing threshold events

6.5.2. Inventory items presence or functional-state events