Using IPMI

Some of our machines have IPMI.

The older Supermicro machines just have IPMI 1.1 and it’s pretty limited and not that usable.

The new (2010) Supermicro machines have IPMI 2.0 and you can do a lot with it.

console tools

Install freeipmi-tools. There are a number of different ipmi tools, each of them implements slightly different things, we’ve been using freeipmi-tools, and they seem to work reasonably well.

All tools support some common flags:

-h or --hostname    hostname or IP of the IPMI processor
-u or --username    username (ADMIN is the IPMI default)
-p or --password    password (ADMIN is the IPMI default)
-P                  prompt for password

remote power: ipmipower

To get an interactive prompt you can run various commands from:

/usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P

or you can run commands directly:
To query the status
/usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P --stat

To turn the machine off and then on

/usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P --off
sleep 10
/usr/sbin/ipmipower --hostname=foo-mp --username=ADMIN -P --on

etc., see the man page for more options.

remote serial console: ipmiconsole

/usr/sbin/ipmiconsole --hostname=foo-mp --username=ADMIN -P

The escape sequence for exiting is 
&.
.

During the bios splash screen, you might need to hit a function key to get at things like the BIOS, or the BBS popup menu. I could get those to work by hitting esc-#, rather than the function keys.

web tools

point your web browser at the IPMI management processor’s hostname/IP.

If the IPMI is on a private network you don’t have direct access to, you can use an ssh tunnel to access it. Run something like

ssh -L4443:ipmihost:443 jumphost.example.com

Then in your browser go to https://localhost:4443/

You can do a few things with just a browser, but for the remote KVM and some other features you must have Sun java installed, currently free java alternatives aren’t enough to make it work.

Here’s what used to work in the squeeze and older days:

Install

sun-java6-bin
sun-java6-jre
sun-java6-plugin

Currently attempting to use icedtea6-plugin or icedtea7-plugin doesn’t work and you get this error

net.sourceforge.jnlp.LaunchException: Fatal: Initialization Error: Could not initialize application. 
        at net.sourceforge.jnlp.Launcher.createApplication(Launcher.java:778)
        at net.sourceforge.jnlp.Launcher.launchApplication(Launcher.java:552)
        at net.sourceforge.jnlp.Launcher$TgThread.run(Launcher.java:889)
Caused by: net.sourceforge.jnlp.LaunchException: Fatal: Application Error: Cannot grant permissions to unsigned jars. Application requested security permissions, but jars are not signed.
        at net.sourceforge.jnlp.runtime.JNLPClassLoader.setSecurity(JNLPClassLoader.java:312)
        at net.sourceforge.jnlp.runtime.JNLPClassLoader.<init>(JNLPClassLoader.java:232)
        at net.sourceforge.jnlp.runtime.JNLPClassLoader.getInstance(JNLPClassLoader.java:357)
        at net.sourceforge.jnlp.runtime.JNLPClassLoader.getInstance(JNLPClassLoader.java:330)
        at net.sourceforge.jnlp.Launcher.createApplication(Launcher.java:770)

Searching with google finds lots of people having problems getting the web based KVM working, it seems that only particular combinations of browser/OS/java version work. That path leads to madness which is why most people recommend using….

IPMIView

Supermicro provides an application called IPMIView, you can download it here. It is also java but appears to work, even on Linux. I’m sure it’s very nice, but it doesn’t allow setting the ports, so that means we need to be running it somewhere with direct network access (either locally or have some sort of VPN to the BMCs). Using an ssh tunnel won’t work since 623 is a privileged port (which would require root to forward it, we don’t want to do that.

Anyway it looks nice but we can’t use it easily….

iKVM

Searching around I found this page complaining about how older versions of IPMIView tried to install their own java on the system and how to run the iKVM java app by hand. I noticed that the ports are set on the command line! So we do this:

  • On the IPMI web interface, go to the configuration→network settings, and change the RMCP
  • In one window start the tunnels for the KVM
    $ ssh -L2623:bnc:623 -L5900:bnc:5900 jumphost.example.com
  • Then from the directory you unpacked the IPMIView application in run
    $ java -Djava.library.path=. -jar iKVM.jar 127.0.0.1 ADMIN ADMIN null 5900 2623 2 0

    Where “ADMIN ADMIN” are your ipmi login/password. This connects to the localhost side of the tunnel using ports 5900 and 2623, which then connect to the BNC ports 5900 and 623.

Virtual Media

The virtual media is limited to a 1.44MB floppy image and using a Windows Share to provide an ISO image.

changing password – IPMICFG

Supermicro provides a command line tool for configuring the IPMI stuff. You can download it here

Check and make sure there is only the ADMIN user and that’s it’s id 2 and then change the password.

root@motmot-pn:~# /tmp/IPMICFG-Linux.x86_64 -user list
Maximum number of Users          : 10
Count of currently enabled Users : 2
User ID | User Name        | Privilege Level | Enable
------- | ---------        | --------------- | ------
      2 | ADMIN            | Administrator   | Yes   
root@motmot-pn:~# /tmp/IPMICFG-Linux.x86_64 -user setpwd 2 <passwd>

The PDF that comes with the tool describes how to use it to do almost everything the web interface can do. (including sensors stuff)

changing password – bmc-config

To change the password of the IPMI device, you first check out its config to a file:

/usr/sbin/bmc-config -h foo-mp -u ADMIN -P --checkout -n foo-ipmi

Now edit the file foo-ipmi with your favorite editor. I found this to be very problematic because the version of the file that is spit out cannot be loaded back in. You have to make a number of adjustments to the file before you can:

For User1, make sure:

  • Enable_User No
  • Lan_Privilege_Limit No_Access
  • SOL_Payload_Access No

For User2:

  • Password
  • Lan_Enable_IPMI_Msgs Yes
  • Lan_Enable_Link_Auth No
  • Lan_Enable_Restricted_to_Callb ack Yes
  • Lan_Privilege_Limit Administrator
  • SOL_Payload_Access Yes

On all the remaining User blocks, make sure:

  • Enable_User No
  • Lan_Privilege_Limit No_Access
  • SOL_Payload_Access No

In Section SOL_Conf make sure:

  • Enable_SOL Yes
  • SOL_Privilege_Level Administrator
  • Force_SOL_Payload_Authenticati on Yes
  • Force_SOL_Payload_Encryption Yes
  • Character_Accumulate_Interval 5
  • Character_Send_Threshold 50
  • SOL_Retry_Count 5
  • SOL_Retry_Interval 10

WARNING: I’ve only included the values above that you need to make sure are set properly, the rest will be fine. I found that many of these were wrong and if you do not set them, you will fail to load the config. For example, the Character_Accumulate_Interval is written as ‘0’ but when you try to load the config it says it is wrong. I found the values from an example config provided by the freeipmi project.

WARNING: with ipmi2 you are supposed to be able to set a 20 character password in the Password20 field. I did that, and loaded the config, and it no longer accepted any password. I believe that you need to use the Password field (not Password20) to set the password. See “Resetting the IPMI password” below for information about how to reset things back to the factory default if you manage to screw it up like I did.

Load that changed file back to the IPMI:

/usr/sbin/bmc-config -h foo-mp -u ADMIN -P --commit --filename=foo-ipmi

resetting an IPMI password

To reset the IPMI password, you need to be on the machine itself that has the IPMI card in it (ie. not from the network).

I am sure there is a way to do this using the packaged tools, but the way I tried first, which succeeded was to do this:

# wget ftp://ftp.supermicro.com/utility/IPMICFG/Linux/IPMICFG-Linux_v1.20.zip
# unzip
# ./ipmicfg-linux.x86.static -fd;./ipmicfg-linux.x86.static -I open lan set 1 password NEWPASSWORD

The ‘set 1’ above is saying that channel 1 is the ethernet interface, on yours it might not be ‘1’, you can find out by doing: ‘ipmitool -I open channel info 1’ which should show something like: Channel Medium Type : 802.3 LAN if it shows ‘Serial/Modem’, try channel 2

resetting the BMC

Sometimes the BMC can hang. You can reset it from the host system using ipmitool (ipmitool package). There are some bugs (#365896, #506934,LP#110992, LP#908112) with the IPMI drivers not automatically loading, so you need to load them first.

modprobe ipmi_devintf
modprobe ipmi_si
# might need to apt-get install ipmitool
ipmitool bmc reset cold

Accessing health data

You can get some basic health information by using one of the two commands:

$ ipmi-sensors -h ip.of.ipmi.device -u ADMIN -P
4: System Temp (Temperature): 31.00 C (-7.00/77.00): [OK]
71: CPU Temp (OEM Reserved): [OEM State = 0000h]
138: FAN 1 (Fan): NA (NA/NA): [Unknown]
205: FAN 2 (Fan): 5025.00 RPM (585.00/29815.00): [OK]
272: FAN 3 (Fan): 5210.00 RPM (585.00/29815.00): [OK]
339: FAN 4 (Fan): 5025.00 RPM (585.00/29815.00): [OK]
406: FAN 5 (Fan): NA (NA/NA): [Unknown]
473: CPU Vcore (Voltage): 0.95 V (0.66/1.41): [OK]
540: +3.3VCC (Voltage): 3.28 V (2.88/3.65): [OK]
607: +12 V (Voltage): 12.35 V (10.60/13.20): [OK]
674: CPU DIMM (Voltage): 1.52 V (1.22/1.78): [OK]
741: +5 V (Voltage): 4.96 V (4.32/5.60): [OK]
808: -12 V (Voltage): -12.29 V (-13.65/-11.51): [OK]
875: VBAT (Voltage): 3.15 V (2.88/3.65): [OK]
942: +3.3VSB (Voltage): 3.26 V (2.88/3.65): [OK]
1009: AVCC (Voltage): 3.28 V (2.88/3.65): [OK]
1076: Chassis Intru (Physical Security): [OK]
1143: PS Status (Power Supply): [Presence detected][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State]

The ipmimonitoring command pulls out the sensor data and instead of reporting them, interprets them to indicate if they are NOMINAL, WARNING or CRITICAL states:

$ ipmimonitoring -h ip.of.ipmi.device -u ADMIN -P
Record_ID | Sensor Name | Sensor Group | Monitoring Status| Sensor Units | Sensor Reading
4 | System Temp | Temperature | Nominal | C | 31.000000 
205 | FAN 2 | Fan | Nominal | RPM | 5025.000000 
272 | FAN 3 | Fan | Nominal | RPM | 5210.000000 
339 | FAN 4 | Fan | Nominal | RPM | 4840.000000 
473 | CPU Vcore | Voltage | Nominal | V | 0.944000 
540 | +3.3VCC | Voltage | Nominal | V | 3.280000 
607 | +12 V | Voltage | Nominal | V | 12.349000 
674 | CPU DIMM | Voltage | Nominal | V | 1.520000 
741 | +5 V | Voltage | Nominal | V | 4.960000 
808 | -12 V | Voltage | Nominal | V | -12.292000 
875 | VBAT | Voltage | Nominal | V | 3.152000 
942 | +3.3VSB | Voltage | Nominal | V | 3.264000 
1009 | AVCC | Voltage | Nominal | V | 3.280000 
1076 | Chassis Intru | Physical Security | Nominal | N/A | 'OK'
1143 | PS Status | Power Supply | Nominal | N/A | 'Presence detected' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State' 'Unrecognized State'

 
stefani
2011-01-28

packages are probably:

sun-java6-bin
sun-java6-jre
sun-java6-plugin

at least, they work for me. but then, i really’d rather not use the web browser just to get to the configuration.
 
 
stefani
2011-05-12

i would like to note that, in my experience, some ipmi-type devices do not work particularly well, or at all, via ssh tunnels.
 
   
taggart
2012-09-04

sun-java6 isn’t in debian anymore as wheezy. There is openjdk and other stuff. I tried installing icedtea-7-plugin and it’s dependencies, but I haven’t gotten things to work (but that might be due to port forwarding).