Category Archives: Solaris

Installing Jupyter Notebooks on Solaris

pip installing jupyterlab may give you the following error (somewhere in the error output anyway!):

The simplest solution I’ve found to this is just to install libzmq. We can pull the latest code from github and install it. I usually install things in /opt but as the pip process will look for libzmq in /usr/local, we’ll install it in there this time.

If you do put it somewhere other than /usr/local you might need to let pkg-config know. E.g.:

Tested on a SPARC-M7 T7-1 kernel zone running 11.4.24.0.1.75.0

Install scipy on Solaris

When trying to install scipy via pip on Solaris you may encounter this error:

There’s a simple way to resolve this but it’s not immediately obvious. BLAS (Basic Linear Algebra Subprograms) and LAPACK (Linear Algebra Something Something…?) are actually provided by the Solaris math and perf libraries, so installing those and setting a couple of environment variables will get scipy and numpy up and running for us on Solaris.

Python 3.7 is in more SRU’s. It’s best practice to use virtualenv to create a virtual environment to work on rather than installing into the system site-packages. I’ve seen a few users make a mess of the packaging system by updating the system python libraries with pip so please just don’t do it 🙂

Tested on a SPARC-M7 T7-1 kernel zone running 11.4.24.0.1.75.0

One-hot encoding labels

Another little snippet that I sometimes forget 🙂

Suppose you have a dataframe with a column that has data in a string format and you need to transform that into a way that a machine learning algorithm can use. One good way is with one-hot encoding which will take the values in the column and create new columns with 1’s & 0’s representing the original data.

Have a look at this dataframe snippet:

prodrev
0Solaris 11.32
1Solaris 11.31
2Solaris 11.44
3Solaris 11.45
4Solaris 11.31

One-hot encoding can be use to transform that:

prodrev0123
0Solaris 11.320.01.00.00.0
1Solaris 11.310.01.00.00.0
2Solaris 11.440.00.01.00.0
3Solaris 11.450.00.01.00.0
4Solaris 11.310.01.00.00.0

And that’s fine for machine learning. Your models and pipelines will just handle it once it added to the original dataframe. But what if you actually want to poke around in the dataframe and use the data yourself? Then it would be really useful to have the label of the column reflect what the data in it actually is. No problem just get the feature names from the encoder and rename the columns on the one-hot dataframe before concating it to the original frame:

prodrevprod_Solaris 11.1prod_Solaris 11.3prod_Solaris 11.4
0Solaris 11.320.01.00.0
1Solaris 11.310.01.00.0
2Solaris 11.440.00.01.0
3Solaris 11.450.00.01.0

Updating Solaris 11 zones behind a proxy

I’ve forgotten this several times now so it’s time I wrote a short post to help me remember in future! In Solaris 11 we use the Image Packaging System to maintain the software on the system. This is written in python and uses libcurl and for non zoned systems setting the http_proxy is sufficient to allow the system to communicate with the repo. Continue reading Updating Solaris 11 zones behind a proxy

Python : generating unittests on the fly

When creating python unit tests you need to create a method in a unittest class. The problem however is when you don’t know until the code is running what these tests will be. In my particular case I need to run a series of tests against a number of Solaris packages. But I won’t know what the packages will be in advance. Continue reading Python : generating unittests on the fly

Python: Searching for a string within a list – List comprehension

The simple way to search for a string in a list is just to use ‘if string in list’. eg:

Continue reading Python: Searching for a string within a list – List comprehension

Tonidoplug Power Supply failure

I’ve been using my tonidoplug as a home NAS server for almost a couple of years now. It’s performed as well as can be expected for 2 mirrored 1.5TB disks attached on a USB hub.

One evening recently though I noticed that it was unreachable. A quick look at the box showed that all was not well, both lights on the network interface were on as was the green light on the box. After trying to reflash the unit I contacted support but they would not replace a nearly 2 year old unit. Though to be fair to them they did offer to sell me a replacement at a discount.

So with the unit dead and no chance of a replacement the ‘void if removed’ sticker was quickly discarded. Here’s the box in its faulted state with the LEDs on:

From experience the most likely thing to die in electronics is the power supply. It clearly isn’t dead here since the lights come on, but a quick check of the outputs with a multimeter showed that what should have been a +5V output was actually +2.5V. Luckily I have a sheevaplug that I use for another home project that uses an equivalent power supply so after a quick swap the Tonido was booting again.

Looking a plug computer forums it appears that there are known issues with some of these power supplies. The Tonidos original power supply was contained in a metal box, as was the original sheevaplug ones that are mentioned in the forums, but my sheevaplug has the circuit board glued to the case with protective black plastic around it. Whether that is to try to reduce the heat build up I can’t say.

Anyway the unit powered back up and the disks and mirror seem to be working fine.

[email protected]:~# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0] sdb1[1]
1465135936 blocks [2/2] [UU]

unused devices:
[email protected]:~#

This does mean that my plans to build a mini-ITX Atom based Solaris ZFS NAS box are now on hold. But at least so are my plans to spend about €500 on it when I can just by a £20 power supply to fix what I have!

Using NAT with Zones

This post originally appeared at http://blogs.sun.com/ford/entry/using_nat_in_the_global

Since sun.com is going away soon (thank a bunch Oracle…) I’m going to mirror it here because I found it useful.

20050927 Tuesday September 27, 2005

Using NAT in the Global Zone

There have been requests to use IPFilter on Solaris 10 as a tool to allow use of networking by non-global zones without allocating an IP address for each zone. The general idea is to use network address translation (NAT) so that packets sent from a non-global zone configured with only a private IP address are modified before leaving the machine to use the global zone’s public IP address. Each zone will be assigned a private IP address that will never be used “on the wire” and has no meaning in the public internet.

I set out to see if this can be accomplished with the existing IPFilter implementation in Solaris 10. My experiments showed that it is possible with some unorthodox system configuration hacks. Special configuration of IP is needed to trick the system into sending this traffic through the right paths so that it is

  • allowed by the zones networking restrictions,
  • passes through IPFilter,
  • is transmitted on the correct interface, and
  • ultimately reaches the proper next-hop router.

My experiment was done using a Solaris 10 (FCS) system with only a DHCP-based connection to the net. My physical network interface was bge0.

This technique requires the use of a block of private IP addresses expressible as an address prefix. I refer to this block as the “imaginary network.” For my experiment I chose 10.10.10.0/24. Two special addresses in the block are allocated: one for the global zone (I chose 10.10.10.1) and one for the imaginary router (I chose 10.10.10.254). It is assumed that the all-zeroes and all-ones node addresses, 10.10.10.0 and 10.10.10.255, are unusable. All other addresses in the block are available for assignment to non-gobal zones (each zone will need one address).

The configuration steps were:

(0) Configure the system (global zone) with normal networking connectivity.

(1) Enable IPFilter.

  ed /etc/ipf/pfil.ap
	/bge/s/#//
	w
	q
	svcadm enable svc:/network/ipfilter
	shutdown -y -i6 -g0

(2) Configure NAT

	echo "map bge0 10.10.10.0/24 -> 0" | ipnat -f -

(3) Assign the global zone’s imaginary address to the physical interface:

ifconfig bge0 addif 10.10.10.1/24 up

(4) Add a default route to the imaginary router.

route  add default 10.10.10.254

(5) Create a fake ARP entry for the imaginary router using the real router’s MAC address.

	netstat -r | grep default
	arp therouter.my.org			# Look up the real router.
	arp -s 10.10.10.254 0:0:c:7:ac:6c	# Use the imaginary router's
						# IP address and the real
						# router's MAC address.

(6) Create one (or more) zone(s) with the same physical net interface as the global zone’s physical interface, and an address on the imaginary network.

zonecfg -z neutral
	...
	add net
	set address=10.10.10.2/24	# a different address for each zone
	set physical=bge0
	end
	commit
	exit

(7) Boot and use the zone(s).

I have only done light testing. Of course it only works with things for which NAT works. If you do something wrong, like forget to enable IPFilter or fail to create the fake ARP entry, you will probably be sending strange packets onto your LAN and network admins may hunt you down and give you dirty looks. The sample commands I list above were typed into this document and may have typos or be rough approximations. Some of the example commands are not persistent across reboot and require other steps to store them persistently. My experiment was on a single user system with simple network connectivity; in a more complex environment the steps to configure it would be very different and it may not work at all.

Conclusion: The technique I used is a hack. I think using it in a production system should be viewed with suspicion, not just because NAT is inherently dubious, but also because of the complex dependencies of the various bits of configuration that are required to make it work. But no doubt it can be useful as an interim technique for some users or applications and maybe a more polished form of the underlying technique can be added to the zones networking tool set eventually.