Today I wanted to introduce Tyrion to the Fedora Planet and anyone who happens to read my blog.
Tyrion is in short a lightweight Func replacement.
I created Tyrion because Func didn't meet some of my needs, specifically the points outlined below.
Tyrion was built using a protocol which is inherently asynchronous. This has lead to lower latency operations and better scalability.
That isn't to say Func couldn't be updated to be asynchronous (using the Twisted XML-RPC libraries for example), but I think current toolsets highly favor protocols which were designed from the beginning to be asynchronous.
(2) Language Agnostic
From a technical perspective Func is also language agnostic (it uses XML-RPC as a transport protocol) and although you could write a client or server in any language, from a pragmatic standpoint you're stuck writing services in Python.
On the other hand Tyrion is written in C++ and each service is a simple executable. This lends itself nicely to writing services in whatever language is most appropriate.
(3) Low memory footprint
Func is written in Python and as such suffers from a dynamic language's higher memory usage.
For example, the following Python application uses ~4.2MB memory on my Fedora 13 box.
#!/usr/bin/env python import time time.sleep(60)
While the complete
tyrion-node application uses ~4.5MB.
For many systems administrators the difference between Func and
memory usage will be negligible, but if you running an infrastructure with
several hundred or several thousand hosts the resource difference can be
A lower memory footprint is also advantageous to companies who heavily utilize
virtualization. If you have 5 servers and you slice those up into 4 virtual
machines each, you've just jumped from 5 to 20 daemons. Lets say Func uses 15MB
memory, that's a 60MB to 300MB jump compared to
tyrion-node's 22.5MB to 90MB.
The protocol that Tyrion uses has built-in support for users. This allows Tyrion to support user specific permissions on a service level and opens up some really cool possibility, such as exposing services to partially trusted users.
In addition because Tyrion uses a process model to run each service, a configuration setting can be defined to specify under which user and group a service is run.
This is a somewhat stupid bullet, but it does have some credibility. Func was created to rely on a hostname/certificate combindation and the assumption that we could communicate to a managed host on a defined port. This setup works great when you have full control over your network.
Unfortunately many of us use external resources (AWS, Linode, Slicehost, etc...) where some or much of the network and infrastructure configuration is out of our control.
Tyrion was built on the XMPP protocol, this means that Tyrion connects to a XMPP server and just listens for events.
With XMPP we also get authentication, an encrypted transport, some decentralization, federation, a solid selection of open source and scalable servers and clients for pretty much every language.
(6) Hot Reloads
The ability to update configuration settings and install services without a daemon restart.
This will briefly overview how Tyrion works.
The Tyrion package has two primary parts, the
tyrion client and the
tyrion-node is daemon which runs on each managed server. It boots with the
host, connects to an XMPP server and listens for service requests.
Below is the general configuration settings for
tyrion-node which are located
[general] acl_path = /etc/tyrion/acl.conf ; ACL configuration file log_path = /var/log/tyrion/node.log ; Logging path log_level = info ; Logging level service_path = /usr/share/tyrion/service ; Service directory [xmpp] jid = email@example.com/tyrion ; Tyrion node JID password = test.node.pass ; Tyrion node password ;server = example.org ; Server address if different from JID domain
node.conf file also lets you define default values for installed services.
[service:org.tyrion.service.bash] timeout = 300 ; Default service timeout (seconds) timeout_lock = false ; Lock default timeout setting user = mike ; Default user to run service as user_lock = false ; Lock default user setting
This tells Tyrion that the
org.tyrion.service.bash service should have a
default timeout of
300 seconds and run as the user
mike. In this case
because we haven't locked the
user option they can be overriden
by the requester.
If you were to look in the
service_path directory you would see an executable
[firstname.lastname@example.org]$ cat /usr/share/tyrion/service/org.tyrion.service.bash #!/usr/bin/env sh /usr/bin/env bash
Obviously this service is a bit redundant and not all that interesting, but serves as a good example of how a service works.
First we'll cover the basics of the
tyrion client. Then we'll use that
information to make a basic service request and follow that request from the
client to the node and back again.
tyrion Client Configuration
[xmpp] jid = email@example.com ; Tyrion node JID password = test.client.pass ; Tyrion node password ;server = example.org ; Server address if different from JID domain [profile:test] jid = firstname.lastname@example.org/tyrion ; Default destination JID service = org.tyrion.service.bash ; Default service type timeout = 300 ; Default timeout user = usertest ; Default user group = grouptest ; Default group
We can see that like the
node.conf file we have our XMPP options under the
xmpp section. Below that we also have a default profile called
to the service section in the
node.conf file this lets us setup some basic
defaults. Here we're defining a destination jid, service, timeout, user and
tyrion Client Options
Additionally if we ran the help option on
tyrion we would see the following:
[email@example.com]$ tyrion --help Usage: tyrion [OPTION]... Example: tyrion -c client.conf Configuration options: -c, --config-file the node configuration file -p, --profile default service options Service options: -j, --jid destination JID(s) -s, --service service type -t, --timeout max service run time -u, --user run service as user -g, --group run service as group Misc options: --debug show debug information
tyrion Client Usage
So an example usage of
tyrion might be:
[firstname.lastname@example.org]$ echo "echo test" | tyrion -c client.conf -p test email@example.com/tyrion (0): test
Here we've run
tyrion with the
client.conf configuration file and the profile
echo test to the
org.tyrion.service.bash service and got the exit
0 and the result
From the other side
tyrion-node sees a request from
to use the service
[org.tyrion.service.bash] firstname.lastname@example.org = true [org.tyrion.service.python] email@example.com = true [org.tyrion.service.ruby] firstname.lastname@example.org = true
After validating that request against the above
acl_path file it runs the
/usr/share/tyrion/service/org.tyrion.service.bash executable and pipes
echo test to stdin.
When the service request completes or reaches the defined timeout it sends the exit code, stdout and stderr back to the client.
tyrion client is really just for testing and one-off hacks. It not suppose
to be used in scripts or anything in production.
Because of this I created
txtyrion, a Twisted-based Tyrion client library.
A simple client application might look like this:
from twisted.internet import defer, reactor import txtyrion class ExampleClient(txtyrion.Client): def success(self, response): print '=' * 80 print ' ID:', response.id print ' JID:', response.jid print ' Code:', response.code print 'Service:', response.service print ' Output:', response.output print ' Error:', response.error def error(self, error): print '=' * 80 print 'Error:', error def done(self, x): print '=' * 80 reactor.stop() def run(self): jid = 'email@example.com/tyrion' command = """ sleep 1 echo bash says hello stdout echo bash says hello stderr >&2 false """ d1 = self.request(jid, 'org.tyrion.service.bash', command) d1.addCallbacks(self.success, self.error) d2 = self.request( jid, 'org.tyrion.service.python', 'print "python says hello"', ) d2.addCallbacks(self.success, self.error) dl = defer.DeferredList([d1, d2]) dl.addCallback(self.done) if __name__ == '__main__': client = ExampleClient( 'firstname.lastname@example.org', 'test.client.pass', debug=True, ) reactor.run()
txtyrion also comes with a node library for creating custom Tyrion nodes (see example).
It's my goal to keep Tyrion simple; to let people utilize it for as much or as little as is appropriate for their situation.
That said I've created a separate project called TyrionHub which will provide a Twisted-based system for managing nodes and a library of Python-based services for systems automation.
The immediate goals for TyrionHub are:
twisted.credand an integration script for ejabbard
I'm currently building TyrionHub with the following technologies:
If anyone is interesting in participating in the development/testing process or
would like to be contacted when TyrionHub is more feature complete, please
contact me at
Everything will be BSD.
I'll be posting a follow-up with a step-by-step explanation on how to setup a basic working version of Tyrion in the next week or two.