Category Archives: computing

Installing Jupyter Notebooks on Solaris

pip installing jupyterlab may give you the following error (somewhere in the error output anyway!):

The simplest solution I’ve found to this is just to install libzmq. We can pull the latest code from github and install it. I usually install things in /opt but as the pip process will look for libzmq in /usr/local, we’ll install it in there this time.

If you do put it somewhere other than /usr/local you might need to let pkg-config know. E.g.:

Tested on a SPARC-M7 T7-1 kernel zone running 11.4.24.0.1.75.0

Install scipy on Solaris

When trying to install scipy via pip on Solaris you may encounter this error:

There’s a simple way to resolve this but it’s not immediately obvious. BLAS (Basic Linear Algebra Subprograms) and LAPACK (Linear Algebra Something Something…?) are actually provided by the Solaris math and perf libraries, so installing those and setting a couple of environment variables will get scipy and numpy up and running for us on Solaris.

Python 3.7 is in more SRU’s. It’s best practice to use virtualenv to create a virtual environment to work on rather than installing into the system site-packages. I’ve seen a few users make a mess of the packaging system by updating the system python libraries with pip so please just don’t do it 🙂

Tested on a SPARC-M7 T7-1 kernel zone running 11.4.24.0.1.75.0

One-hot encoding labels

Another little snippet that I sometimes forget 🙂

Suppose you have a dataframe with a column that has data in a string format and you need to transform that into a way that a machine learning algorithm can use. One good way is with one-hot encoding which will take the values in the column and create new columns with 1’s & 0’s representing the original data.

Have a look at this dataframe snippet:

prodrev
0Solaris 11.32
1Solaris 11.31
2Solaris 11.44
3Solaris 11.45
4Solaris 11.31

One-hot encoding can be use to transform that:

prodrev0123
0Solaris 11.320.01.00.00.0
1Solaris 11.310.01.00.00.0
2Solaris 11.440.00.01.00.0
3Solaris 11.450.00.01.00.0
4Solaris 11.310.01.00.00.0

And that’s fine for machine learning. Your models and pipelines will just handle it once it added to the original dataframe. But what if you actually want to poke around in the dataframe and use the data yourself? Then it would be really useful to have the label of the column reflect what the data in it actually is. No problem just get the feature names from the encoder and rename the columns on the one-hot dataframe before concating it to the original frame:

prodrevprod_Solaris 11.1prod_Solaris 11.3prod_Solaris 11.4
0Solaris 11.320.01.00.0
1Solaris 11.310.01.00.0
2Solaris 11.440.00.01.0
3Solaris 11.450.00.01.0

Django Q function

When querying objects in django the simplest way is to use the filter() query.

If we have a set of systems for example and we want to filter out those that have a SPARC architecture we could do:

And if we wanted to narrow that down to only sparc machines that use NIS for name resolution we could do:

But that is narrowing down the selection. It’s an AND. What were asking for is systems that are a SPARC _AND_ use NIS. And although we can use exclude() to reject some results it’s still and AND operation. What about an OR query?

What if we wanted to select systems that had either a T5 or a T7 processor? Then we need Q queries.

Q queries take the same form of argument as filter() & exclude() etc. but can be combined using OR operators. Here’s the Q query for our model that looks at the cpu_implementation for T5 or T7:

To build our query we just use this Q expression in a filter:

Python: Creating a singleton (to control an Arduino)

A while ago I was writing a tool in python to communicate with a sensor on an Arduino. The idea was to have a class for the Arduino that would hold the connection and handle the reading and writing to the serial port. However that way I needed to have the object maintain the connection and the open COM port (this was on windows), but this meant that I could not have multiple objects. Why? Because you can only have one connection to a COM port. The solution – use a Singleton design pattern.

Continue reading Python: Creating a singleton (to control an Arduino)

Detecting Meteors from radio reflections

When I was in school, I remember being told that it was possible to tune an FM radio do a free frequency low in the band and during a meteor storm you could ‘hear’ when meteors struck the atmosphere as the FM signal from distant radio stations would be reflected back down from the ionised meteor trail. This may have been possible as back then there were plenty of high powered FM stations from Eastern Europe using a lower portion of the spectrum and receivers often went below the 88.5 that is the limit on the dial now. But I never heard anything. Over the years as my interest in meteors and radio was peeked I looked into it again, but the number of stations to use was dwindling and there were few artificial sources that could produce an audible ping. Continue reading Detecting Meteors from radio reflections

Updating Solaris 11 zones behind a proxy

I’ve forgotten this several times now so it’s time I wrote a short post to help me remember in future! In Solaris 11 we use the Image Packaging System to maintain the software on the system. This is written in python and uses libcurl and for non zoned systems setting the http_proxy is sufficient to allow the system to communicate with the repo. Continue reading Updating Solaris 11 zones behind a proxy

Helping you manage you Solaris 11 support updates

… or Overcoming Your Fear of Repositories as the OTN Garage puts it! My colleagues Glynn Foster, Pete Dennis and I have put three articles on Oracle Tech Network that we hope will help you keep your Solaris 11 systems up to date. Here’s why these documents may be of interest to you administering Soalris 11.

Continue reading Helping you manage you Solaris 11 support updates