Firstly you need to disable nwam.

svcadm disable svc:/network/physical:nwam
svcadm enable svc:/network/physical:default

Check that there are no nwamd process left running.

To set your address you need to see what interface you have availble. On a typical install that already has a working DHCP interface you can see what is already in use by:

# ipadm show-addr
ADDROBJ           TYPE     STATE        ADDR
lo0/v4            static   ok           127.0.0.1/8
net0/_b           dhcp     ok           12.34.56.78/21
# 

So to set your static IP you would just do:

# ipadm create-addr -T static -a 12.34.56.78/21 net0/v4

However you may notice that this fails:

# ipadm create-addr -T static -a 12.34.56.78/21 net0/v4
ipadm: cannot create address: Persistent operation on temporary object

The issue here is that net0 has been created as a temporary object by the system. Which you can verify by noticing the missing entries on the PERSISTENT column of the output of show-if:

# ipadm show-if -o all
IFNAME     CLASS    STATE    ACTIVE CURRENT       PERSISTENT OVER
lo0        loopback ok       yes    -m46-v------  46--       --
net0       ip       down     no     bm46--------  ----       --

In this case delete and re-create the interface, then set your address:

 # ipadm delete-ip net0
# ipadm create-ip net0
# ipadm create-addr -T static -a 12.34.56.78/21 net0/v4

The last thing you need to do is set your default route which is is done by:

# route -p add default 12.34.56.1

Next you may need to set up your system as a DNS client. This is now done by SMF and not by editing files by hand. To set DNS servers:

root@sionnach:/home/albwhite# svccfg -s dns/client
svc:/network/dns/client> listprop config
config                      application
config/value_authorization astring     solaris.smf.value.name-service.dns.client
config/domain              astring     ie.oracle.com
svc:/network/dns/client> setprop config/nameserver = (12.34.56.17 192.168.82.46)
svc:/network/dns/client> listprop config
config                      application
config/value_authorization astring     solaris.smf.value.name-service.dns.client
config/domain              astring     ie.oracle.com
config/nameserver          net_address 12.34.56.17 192.168.82.46
svc:/network/dns/client> exit
root@sionnach:/home/albwhite# svcadm refresh dns/client
root@sionnach:/home/albwhite# svcadm restart dns/client

Similarly to set the resolv.conf entries you would do:

svccfg -s name-service/switch
> setprop config/host = "files dns"

Official documentation: http://docs.oracle.com/cd/E23824_01/html/821-1458/gdyrf.html

Thankfully python allows to to add a method to a class on the fly:

>>> class test:
...     pass
...
>>> def foo(self):
...     print "hello, world"
...
>>> dir(test)
['__doc__', '__module__']
>>> test.foo=foo
>>> dir(test)
['__doc__', '__module__', 'foo']
>>> bar=test()
>>> bar.foo()
hello, world
>>>

Here we initially create an empty class ‘test’ and a function ‘foo’. Then add the function ‘foo’ to the class ‘test’, so that when we create a ‘test’ object, we can call the ‘foo’ method in it.

We use the same trick to add a unittest test to a unittest class:

>>> import unittest
>>> class mytest(unittest.TestCase):
...     pass
...
>>> def test_testcase(self):
...     self.assertEqual("hello","hello")
...
>>> suite = unittest.TestLoader().loadTestsFromTestCase(mytest)
>>> unittest.TextTestRunner(verbosity=2).run(suite)

----------------------------------------------------------------------
Ran 0 tests in 0.015s

OK
<unittest.runner.TextTestResult run=0 errors=0 failures=0>
>>> mytest.test_testcase=test_testcase
>>> unittest.TextTestRunner(verbosity=2).run(suite)

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
<unittest.runner.TextTestResult run=0 errors=0 failures=0>
>>> suite = unittest.TestLoader().loadTestsFromTestCase(mytest)
>>> unittest.TextTestRunner(verbosity=2).run(suite)
test_testcase (__main__.mytest) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
<unittest.runner.TextTestResult run=1 errors=0 failures=0>
>>>

>>> list=['a cat','a dog','a yacht']
>>> string='a cat'
>>> if string in list:
...     print 'found a cat!'
...
found a cat!
>>>

But what if you need to search for just ‘cat’ or some other regular expression and return a list of the list items that match, or a list of selected parts of list items that match.
Then you need list comprehension.

>>> import re
>>> list=['a cat','a dog','a yacht','cats']
>>> regex=re.compile(".*(cat).*")
>>> [m.group(0) for l in list for m in [regex.search(l)] if m]
['a cat', 'cats']
>>> [m.group(1) for l in list for m in [regex.search(l)] if m]
['cat', 'cat']
>>>

Lets work through that. Firstly we’re going to use the regular expression library so we import that. re.compile lets us create a regular expression that we can later use for search and matching.

Now lets look at a simpler comprehension line:

>>> [m for l in list for m in [regex.search(l)]]
[<_sre.SRE_Match object at 0x10cfbb0a8>, None, None, <_sre.SRE_Match object at 0x10cfbb6c0>]

This is creating a list of the results of running regex.search() on each item in l.

Next we want to exclude the None values. So we use an if statement to only include the not None values:

>>> [m for l in list for m in [regex.search(l)] if m ]
[<_sre.SRE_Match object at 0x10cfbb6c0>, <_sre.SRE_Match object at 0x10cfd0be8>]
>>>

Lastly rather than returning just a list of the m’s, which are RE match objects, we get the information we want from them.

>>> [m.group(0) for l in list for m in [regex.search(l)] if m]
['a cat', 'cats']
>>> [m.group(1) for l in list for m in [regex.search(l)] if m]
['cat', 'cat']
>>>

Very useful if you have say a list of Solaris 11 package fmri’s and want to extract a set of package names based on a regular expression of their version number for instance

Handy when you screw up boot environment activation

On x86 you can just select the be you want in grub.

{0} ok boot -L
Boot device: /pci@400/pci@0/pci@8/scsi@0/disk@0,0:a  File and args: -L
1 Oracle Solaris
2 solaris-1
Select environment to boot: [ 1 - 2 ]: 2

To boot the selected entry, invoke:
boot <root-device> -Z rpool/ROOT/solaris-1

Program terminated
{0} ok boot /pci@400/pci@0/pci@8/scsi@0/disk@0,0:a -Z rpool/ROOT/solaris-1

One evening recently though I noticed that it was unreachable. A quick look at the box showed that all was not well, both lights on the network interface were on as was the green light on the box. After trying to reflash the unit I contacted support but they would not replace a nearly 2 year old unit. Though to be fair to them they did offer to sell me a replacement at a discount.

So with the unit dead and no chance of a replacement the ‘void if removed’ sticker was quickly discarded. Here’s the box in its faulted state with the LEDs on:

From experience the most likely thing to die in electronics is the power supply. It clearly isn’t dead here since the lights come on, but a quick check of the outputs with a multimeter showed that what should have been a +5V output was actually +2.5V. Luckily I have a sheevaplug that I use for another home project that uses an equivalent power supply so after a quick swap the Tonido was booting again.

Looking a plug computer forums it appears that there are known issues with some of these power supplies. The Tonidos original power supply was contained in a metal box, as was the original sheevaplug ones that are mentioned in the forums, but my sheevaplug has the circuit board glued to the case with protective black plastic around it. Whether that is to try to reduce the heat build up I can’t say.

Anyway the unit powered back up and the disks and mirror seem to be working fine.

root@TonidoPlug:~# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0] sdb1[1]
1465135936 blocks [2/2] [UU]

unused devices:
root@TonidoPlug:~#

This does mean that my plans to build a mini-ITX Atom based Solaris ZFS NAS box are now on hold. But at least so are my plans to spend about €500 on it when I can just by a £20 power supply to fix what I have!

Since sun.com is going away soon (thank a bunch Oracle…) I’m going to mirror it here because I found it useful.

Tuesday September 27, 2005

Using NAT in the Global Zone

There have been requests to use IPFilter on Solaris 10 as a tool to allow use of networking by non-global zones without allocating an IP address for each zone. The general idea is to use network address translation (NAT) so that packets sent from a non-global zone configured with only a private IP address are modified before leaving the machine to use the global zone’s public IP address. Each zone will be assigned a private IP address that will never be used “on the wire” and has no meaning in the public internet.

I set out to see if this can be accomplished with the existing IPFilter implementation in Solaris 10. My experiments showed that it is possible with some unorthodox system configuration hacks. Special configuration of IP is needed to trick the system into sending this traffic through the right paths so that it is

• allowed by the zones networking restrictions,
• passes through IPFilter,
• is transmitted on the correct interface, and
• ultimately reaches the proper next-hop router.

My experiment was done using a Solaris 10 (FCS) system with only a DHCP-based connection to the net. My physical network interface was bge0.

This technique requires the use of a block of private IP addresses expressible as an address prefix. I refer to this block as the “imaginary network.” For my experiment I chose 10.10.10.0/24. Two special addresses in the block are allocated: one for the global zone (I chose 10.10.10.1) and one for the imaginary router (I chose 10.10.10.254). It is assumed that the all-zeroes and all-ones node addresses, 10.10.10.0 and 10.10.10.255, are unusable. All other addresses in the block are available for assignment to non-gobal zones (each zone will need one address).

The configuration steps were:

(0) Configure the system (global zone) with normal networking connectivity.

(1) Enable IPFilter.

  ed /etc/ipf/pfil.ap
	/bge/s/#//
	w
	q
	svcadm enable svc:/network/ipfilter
	shutdown -y -i6 -g0

(2) Configure NAT

	echo "map bge0 10.10.10.0/24 -> 0" | ipnat -f -

(3) Assign the global zone’s imaginary address to the physical interface:

ifconfig bge0 addif 10.10.10.1/24 up

(4) Add a default route to the imaginary router.

route  add default 10.10.10.254

(5) Create a fake ARP entry for the imaginary router using the real router’s MAC address.

	netstat -r | grep default
	arp therouter.my.org			# Look up the real router.
	arp -s 10.10.10.254 0:0:c:7:ac:6c	# Use the imaginary router's
						# IP address and the real
						# router's MAC address.

(6) Create one (or more) zone(s) with the same physical net interface as the global zone’s physical interface, and an address on the imaginary network.

zonecfg -z neutral
	...
	add net
	set address=10.10.10.2/24	# a different address for each zone
	set physical=bge0
	end
	commit
	exit

(7) Boot and use the zone(s).

I have only done light testing. Of course it only works with things for which NAT works. If you do something wrong, like forget to enable IPFilter or fail to create the fake ARP entry, you will probably be sending strange packets onto your LAN and network admins may hunt you down and give you dirty looks. The sample commands I list above were typed into this document and may have typos or be rough approximations. Some of the example commands are not persistent across reboot and require other steps to store them persistently. My experiment was on a single user system with simple network connectivity; in a more complex environment the steps to configure it would be very different and it may not work at all.

Conclusion: The technique I used is a hack. I think using it in a production system should be viewed with suspicion, not just because NAT is inherently dubious, but also because of the complex dependencies of the various bits of configuration that are required to make it work. But no doubt it can be useful as an interim technique for some users or applications and maybe a more polished form of the underlying technique can be added to the zones networking tool set eventually.

The client in question had needed to be rebooted earlier in the day. The server had had no changes made to it. The problem only started occurring after the reboot of the client.

We had an NFS directory on the server that the client’s subnet had access to. However when we tried to mount the directory (via the automounter) it failed with a permission denied error. I should point out at this stage that the client is multihomed, that it it has several network interfaces onto different subnets and only one of these was allowed to access the share.

A little snooping showed that while the requests were mostly going out from the correct interface, some packets were coming out on an interface for another subnet. This shouldn’t occur since there is only one route packets can take out and its through the default router on the main interfaces subnet. It turns out however that we have 3 default routers on this machine – all active. This is likley a remnant of how the machine was installed, we copy in a defaultrouter file with multiple entries and since hosts usually only have one interface the rest get ignored. In the case of this system though the interface had been manually added over time and the routes were not activated, until the reboot then they all became active.

This meant that the system had 3 interfaces to contact the server on, but that the server would only authorise one of these interfaces. Since the interfaces had routes to the server the portmapper (rpcbind) was happily mapping ports on interfaces that we’d rather it didn’t.

Delete the routes, and delete the defaultrouter entries, restart rpcbind, and all works again.

When you want to display something from one unix box to another there are two ways of doing it.
1. Use ssh. You need to use ‘ssh -X’ specifically which will allow X11 forwarding. ssh takes care of setting the DISPLAY variable for you. You also need to ensure that the server is configured for X forwarding:

# X11 tunneling options
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes

This just works if you are a regular user, but if you need to display something as another user then you run into problems. For example you may need to run a gui installer that can only run as root.
So heres how to get it to work as root.
Firstly, as your normal user get your DISPLAY variable and the corresponding xauth entry. This can be a little tricky

$ echo $DISPLAY
localhost:10.0

Now search the xauth list for the corresponding entry. If you got localhost above, then search for the actual hostname. On some systems ‘xauth list $DISPLAY’ may work fine, others you’ll need to search:

$ xauth list|grep :10|grep server
server/unix:10  MIT-MAGIC-COOKIE-1  99410f7406b0f93e6117ae55a72a7eca

Next become the root user, or whatever user you need to be.
Set the display variable and add the xauth entry:

# DISPLAY=localhost:10
# export DISPLAY
# xauth add server/unix:10  MIT-MAGIC-COOKIE-1  99410f7406b0f93e6117ae55a72a7eca

You should now be able to fire up an xterm on the system as the new user and it will appear on your desktop.

2. Use xhosts. This is the more traditional way, but does have a gotcha for newer versions of solaris, which are more secure than previously.
On your desktop do:

$ xhost +
access control disabled, clients can connect from any host
$ echo $DISPLAY
:0.0

Now log onto the remote system and set the display variable and try an xterm. It will probably fail:

$ DISPLAY=desktop:0.0
$ export DISPLAY
$ xterm
xterm Xt error: Can't open display: desktop:0.0
$ 

Whats happening here is that although you have allowed the xserver access control to accept connections from any host, the X server itself isnt actually listening for connections. To change that you need to do the following on your desktop:

 svccfg -s svc:/application/x11/x11-server setprop options/tcp_listen = true 

Then restart the Xserver.
This method will work for all users, so you can log in, become root, then just set your display.

3 ok there are alternative methods if you are really stuck. For example start up a VNC session on the server and view it on your desktop. Though thats only worth doing if you need someone else to be able to view what you are doing.

Docs.sun.com has some information on the differences between the two formats http://docs.sun.com/app/docs/doc/817-5093/disksconcepts-47761?l=en&a=view

But suffice to say that you cant install Solaris on an EFI disk, you need to change it to an SMI label. Thats done using ‘format -e’.

Select the label option, and then SMI Label. Once you have done that you can check the partition table as normal and your install can progress.

(II) LoadModule: “vboxvideo”
(II) Loading /usr/X11/lib/modules/drivers/vboxvideo_drv.so
dlopen: ld.so.1: Xorg: fatal: relocation error: file /usr/X11/lib/modules/drivers/vboxvideo_drv.so: symbol resVgaShared: referenced symbol not found
(EE) Failed to load /usr/X11/lib/modules/drivers/vboxvideo_drv.so
(II) UnloadModule: “vboxvideo”
(EE) Failed to load module “vboxvideo” (loader failed, 7)

Alan Coopersmith helped me work out what was going wrong here. When you install the guest additions on virtualbox they create an xorg.conf file. The existing conf file I had from build 127 and VB 3.1.1 worked fine, and build 127 with vb 3.1.4 worked fine. But due to changes around Xorg 1.7 things broke when I upgraded opensolaris to build 132 – even though virtualbox has support for this.

The solution?

• Remove the guest additions package.
• Remove the xorg.conf file
• Re-install the guest additions
• Reboot

All works fine now!