geekscape’s blog

Continuing on from the initial launches of a Class C rocket, the Lunar Numbat build team has been making steady progress towards launching more sophisticated avionics, along with an audio/video feed, as part of a Class G rocket.

A crucial part of the journey has been, not just creating the hardware and software, but bringing more capability and experience to the effort, in the form of organization, equipment and most importantly ... people with specialist skills.

For a long time, we've been contemplating the need for a shared space or workshop, expensive electronics equipment and mechanical manufacturing capabilities beyond that which individuals can typically justify or afford.  These sorts of facilities are available in universities and specific types of government or commercial organizations, but they are much less accessible to smaller communities of individual developers.

At the end of March 2009, a Wired magazine article propelled the concept of HackerSpaces into the popular zeitgeist.  So, it was an natural step to create the first HackerSpace in Melbourne, Australia, aka the "Connected Community".

The Connected Community HackerSpace was formed and operates independently from the Lunar Numbat project.  A HackerSpace is much broader in scope than even a space technology project, to the point of hacking any object in any domain in any conceivable fashion is fair game.  Fortunately, there is significant overlap in the types of people involved in both groups, the skills and equipment required and the artifacts created.

An advantage of the HackerSpace is that it attracts a larger community of hobbyists and professionals.  These people may not be as focused or as passionate about the research and development of space technology as the Lunar Numbat team.  Although, sometimes they are even more passionate.  However, their skills can directly or indirectly contribute to the Lunar Numbat effort, due to the deeply embedded culture of sharing.  This collaboration is enhanced by both groups utilizing commodity, open-source hardware and software.  The power of open-source hardware and software communities is evident in the growing number of developers using the Arduino platform or BeagleBoard for increasingly sophisticated projects.

Since it's inception, the HackerSpace has been meeting regularly and working on a variety of projects. Many of the projects have been founded by HackerSpace members, often initiated prior to the HackerSpace and operating independently.  In addition, the HackerSpace is introducing new members to those projects and bringing them up-to-speed with fundamental skills, such as PCB design and manufacture.  New members also bring additional skills such as amateur radio experience, CNC milling machine design, construction using composite materials or hard-core hardware and software engineering ... and, even rocketry.

The Lunar Numbat project has benefited from this infusion of new blood and conversely, the Class G rocket development has been one of the projects around which HackerSpace members can gravitate and apply their skills.

Project progress and technical discussions have been captured on the Connected Community HackerSpace web-site and project artifacts placed in social coding repositories, such as the popular GitHub, as follows ...

• Class G rocket project
• Class G rocket hardware design
• Aiko: Modular event-driven software framework

A key outcome will be to deliver re-usable hardware and software components.  In part, this is driving us to create general purpose frameworks, like Aiko, which provide a modular, event-driven abstraction on top of specific hardware / software platforms, like the Arduino.  This is so that we can produce high-quality embedded applications and software components that encapsulate a given set of hardware devices.

In the longer term, this collaboration between the Connected Community HackerSpace and Lunar Numbat, along with the resulting re-usable hardware and software artifacts, provides a strong foundation for undertaking the next phases of Lunar Numbat beyond the current Class G rocket launch milestone.

[Posted on the Lunar Numbat Google Group]

A brief summary of the talks I attended today ...

My LCA 2009 Flickr photostream

NDB / MySQL Cluster by Monty Taylor (Sun Microsystems) mordred@sun.com
- URLs ...
  - https://launchpad.net/ndb-bindings
  - ndb-connectors@lists.mysql.com
  - http://lists.mysql.com
  - IRC #mysql-ndb at freenode.net
- High availability, 5 9's (99.999%)
- Sub-second fail-over
- Configurable redundancy, e.g. number of replicas
  - Best to use just 2 replicas, using more isn't necessarily a good thing
- High performance through parallelism
- For simple primary key look-ups as fast as memcached
  - But, harder to configure
  - Doesn't implement foreign keys
*** Fully asynchronous, includes an Event API ***

- Data and indices kept in main memory and check-pointed to disk
- Share nothing, commodity PCs, commodity interconnects, no expensive shared disk
  - No single point of failure
- Originally a project at Ericsson, then MySQL and now Sun Microsystems
- Hash (MD5) on the primary key and split across systems accordingly
- Storage Engine API (per table configuration)
  - Like a Virtual File System
  - Different implementation, e.g. InnoDB, MyISAM, NDB, Falcon
  - NDB doesn't exist in Drizzle yet
  - NDB only really exists in the MySQL cluster fork
- mysqld talks to cluster of NDB nodes, using NDB wire protocol
- Using NDB directly is roughly 5 times faster than SQL
- With 16 nodes has achieved ...
  - Using MySQL: 120,000 transactions per second
  - Using NDB 5.1 directly: 500,000+ transactions per second
- Complete NDB API ...
  - Primary Key operations: readTuple, writeTuple, deleteTuple
  - Index Scan - readTuples
  - Table Scan - readTuples
  - Unique Index Scan - readTuples
  - All other operations can be broken down into combinations of these
- NDB API / Bindings for Java (most complete), Python, Ruby, Perl, C#, Lua
- Require ...
  - MySQL installed with libs and headers
  - MySQL 5.1 CGE 6.2 required (and 6.3.14 or 6.3.16 will also work)
- C++ API (libndbclient) wrapped with SWIG
  - Java bindings (NDBJ) or JNI via SWIG
  - Java NDB object is not thread-safe, need to have one NDB instance per thread
- REST API: http://code.google.com/p/mod-ndb/
  - Returns: JSON, XML, YAML

Transcoding in MythTV by Paul Wayper

- Capture analogue signal can result in significant storage requirements
  - Good idea to transcode from RTJPEG into MPEG-4
- Remove "cut regions", e.g. advertisements, etc.
  - Editing using "e" key whilst watching
  - "z" to import commercial flagged regions
- Transcoding ...
 - Removes previously set "cut regions", but not "flagged regions"
 - Downsize physical area
 - Recompress using MPEG-4 (fixed bit rate with quality settings) and downsample audio
 - Exporting to other media
   - Using nuvexport to reformat and export to external storage
     - But, easier to just add storage and kept video stored in MythTV
   - User jobs are harder to configure, separate script, can be performed automatically
     after transcoding, environment variables for parameters
     - An example: export to mobile phone 3gpp files
- Ended up talking a lot about his custom notebook cover made from laser-cut wood veneer

Panel on Geek Parenting by LinuxChix

- Q: How did you start out as a "geek parent: ?

  A: Reading books on parenting
  A: Lessons learned from parent's parenting
  A: Listen to all the details and throw it all away apart from the "nuggets of wisdom"
  A: Not something you can prepare for intellectually, because it's not an intellectual process
  A: Freaking out from total information overload. Avoid taking on too much information from outside
  A: Funniest book: "Perfect parents" (history of parenting)

- Q: What type of teaching, before your kids started formal education ?

  A: Reading to them and music
  A: The simple things that we all take for granted, e.g. language and mathematics
  A: Lots of group activities, getting them out in the open
  A: Parent-child relationship goes both ways, e.g. parent also learns from child
  A: Allow child's simple explorations become opportunities for "finding out about the world"
  A: Read "thousands" of books to kids, counting games (with numbers)
  A: Looking for toys with some sort of "creative construction" element, e.g. Lego
  A: Less interest in games with fixed or limited outcomes, rather than more open-ended
  A: Capacity to learn driven by motivation
  A: Engage them in activities: physical, social, intellectual, artistic
  A: Know when to "stop" ... when some activity isn't working
  A: Know when not to "stop" ... don't let them learn to give up when things get hard

- Q: Double challenge for parents of girls, encouraging interest in maths and science
  A: Social stigma associated with girls and geeky, technical subjects ... still true ?
  A: Are toys becoming more segregated into boys and girls ?
  A: Still opportunities for balance between girl and boy interests.

- Q: Juggling parenting and your career ?
  A: Divorced, by self, entrepreneur ... organised with a regular routine, balance, prioritise
  A: Important to spent more time at home with your children, if at all possible
  A: Having a full-time, start at home Mum can be good, if at all possible
     But, need to ensure that you are very engaged as a parent when you are at home

  Gever Tulley: TED Talk: 5 Dangerous things your kids should do !
  http://www.ted.com/index.php/talks/gever_tulley_on_5_dangerous_things_for_kids.html

System Administration Lightning Talks

VESPER (Virtual Embraced Space ProbER) by Sungho Kim
- http://sysadmin.miniconf.org/lightning09.html#01
- Uses KProbes (Kernel Dynamic Probes = automated breakpoint, small C functions)
  - http://sourceware.org/systemtap/kprobes
- Gathers information from the guest operating system under test

Mondorescue via PXE by Simon Lyall
- http://sysadmin.miniconf.org/lightning09.html#03
- http://www.mondorescue.org and http://trac.mondorescue.org
- Documentation: /usr/share/docs/HOWTO.pxe (?)
- Backup and Recovery tool
- Creates bootable images of running servers
- Intended for system and appications
- Not for backing up all your data, e.g. not database, not video
- # mount ...; mondoarchive ...; umount ...
- Full or incremental back-ups
- Use with PXE for booting and restoring system
- Also good for cloning machines
- Read the Trac FAQ (more up-to-date) after the main FAQ

Think by Frank Sainsbury
- http://sysadmin.miniconf.org/lightning09.html#04
- We are paid to "think"
- Frank has been at the "rough end" of some pretty poor thinking
- Works for someone who wants everything and wants it yesterday
- Project: Places SMS messages into an Oracle database
- Based on Windows 2003 server and Oracle database
- Tactical thinking: How to fix the problem
- Strategic thinking: Longer term ... but no-one had thought about that, e.g. keep it simple
- Should perform strategic thinking, even if it is a (so called) simple task
- Security has become a bit part of what we do these days
- Running 2 versions of Apache, versions of Java, JavaScript, Mono and many, many other things !
- Need shared access to good Oracle database models for a specific domain, improve implementation time
- Caught out by RedHat ES deployment issue
- Need all Linux hackers / system administrators to *think*, collaborate and do it better !

System administration - a programming perspective by Devdas Bhagat
- http://sysadmin.miniconf.org/lightning09.html#02
- Need to think in terms of multiple systems, not single systems
- Need to have centralised configuration system management
- System administrator also needs to be a programmer to solve the problems
- Opposing view: Centralised configuration can be overkill
- Centralised deployment management ?
- Use "puppet" ... http://reductivelabs.com/trac/puppet/wiki/PuppetIntroduction
- Test Driven Systems Administration ?
- System Administrators need to work much more closely with Developers, particularly for production environment
- Developers need to test against an environment that is as close as possible to the production environment

Large CPU masks by Rusty Russell

- Kernel primitives ...
  - Runtime NR_CPUS and cpus_weight_nr() and for_each_cpu_mask_nr(...)
  - set_cpus_allowed_ptr()
  - Magic declaration and allocation macros
  - Returns pointer to cpumask for specified node
- These primitives are applied on an "as needed basis" ,,,
- And trial-and-error, only fix when broken
- What if distributions set CONFIG_NR_CPUS=4096, even if you only have 2 CPUs ?
- 2.6.29 aim is to remove (almost) all cpmasks from x86 stack
- CONFIG_CPUMASK_OFFSTACK: currently only set by x86/64 MAXSMP or via DEBUG_CPUMASKS
- We do have a typedef "cpumask_var_t" and also "alloc_cpumask_var(&var, fp)"
- Need to provide new kernel API, switch all code across, then deprecate the old kernel API
  - Usually takes at least one cycle of kernel development
- 2.6.30 aim is to remove all cpumasks and old operators from everywhere on x86
- Fairly good set-up for "static" per-cpu variables ...
  - DEFINE_PER_CPU(type, name)
  - __get_cpu_var(name)
  - get_cpu_var(name), put_cpu_var_name(), per_cpu(var, cpu)
- Also have "temporary" system (now 5 years old) for "dynamic" per-cpu variables ...
  - Mirror "static" per-cpu variables, but uses alloc / free
- Do have a real allocator inside the module code, but never exposed because per-cpu space is severely limited
Longer term: Growing per-cpu areas ?
  - Virtual contiguous layout scheme ?
  - Then networking code can move back into using per-cpu areas

Presence Awareness for MythTV by Jonathan Oxer
- SuperHouse: Channel 31 in Melbourne, later in the year
- Blog: jon.oxer.com.au
- Flames: jon@oxer.com.au
- Web: http://superhouse.tv
- Using Arduino + PHP code to play/pause MythTV
- Multi-room MythTV and follow-me
- Presence loss: query location, parse and log -> Last event for a particular person
- Presence gain: read log, change channel and play from given point in movie
  - Problem MythTV back-end doesn't provide play video from given point in movie
- Use "zoneminder" and security cameras
- Facial recognition: Intel video library "opencv"

MythTV Lightning Talks

MythTV usability: Customisation and tweaking versus the Wife Acceptance Factor !
- HTPC - Home Theatre PC
  - Two vague terms crammed together ?!?
  - HTPC - Customisation ?
  - MythTV - Convergence ?
  - PVR - Convenience ?
- Not having to mess with a collection of video tapes or DVDs
- Customizations: Best image quality, scaling
- Windows: FFDShow http://www.avsforum.com ... difficult to configure, too many hours required
- Good versus Bad customisation
  - Good: Learning remote, Useful jobs (copy and transcode, burn to DVD), MythTV menus
  - Useful: Remote admin: ssh -L 9999:localhost:80 host

Linux kernel Panel

- Worst bug introduced into the kernel ?
  - Memory leak in the ICMP stack, took a year to discover
  - Removed semicolon (?) which broke part of the build for 6 weeks, before it was discovered
  - Broke the build in Linus' tree
  - Broke code turning off barriers, which wasn't possible to test, but broke for someone in the field
  - Forgot some parenthesis so that a divide-by-two became multiple-by-two, lead to allocation
    growing too fast.  Took 2 years to discover.

- How did you become a kernel hacker ?
  - After fixing lots of kernel bugs (in Debian)
  - Was Mac OS hacker, started playing with Linux, noticed it didn't run so well, knew hardware
    pretty well, and slowly started hacking kernel more and more
  - Started hacking on hand-rolled BSD kernels in embedded systems.  Moved across to working on XFS.
  - Started working at Sequent on MPP systems.  Hired by IBM, started working on broken Linux kernel
    things
  - Started kernel hacking in 1996, was sick of working on minority O.S.
    Linux looked like the next bug thing.  First patch was to ISO9960 file system

- What development environments and work flow process do you use ?
  - VI, GIT, make and shell scripts
  - Emacs (minimal features)
  - Anything that runs in an xterm, e.g. VIM, user mode Linux, sometimes Linux on ESX
  - KVM or VMWare to test ISO of distribution
  - Make and test on real hardware, due to using drives for device specific hardware
  - Would be useful to have tool to many KVM images
  - Use "quilt" for work-flow
  - Should use "guilt", which is "quilt" on GIT :)
- When will you be finished [as a kernel hacker] ?

  - When I've drunk away the less of my brain cells
  - When it stops being fun
  - When I can buy my very own goat farm [laughter]

- Worst part of being a kernel hacker ?
  - There is no holiday, it's 24x7 !  Always someone asking a question
  - Having the control-D key wearing out on all my keyboards
  - Too much stuff to keep track off
  - If you aren't deleting your email intray messages, you're not a kernel hacker
  - Can't keep up with all of Linux Kernel email list messages
  - No-one person has a full grasp on the whole kernel ... but, it's getting worse
  - Other people breaking your stuff
  - Coming back from vacation ... with pages and pages of messages
  - Other people touching stuff [laughter]
  - Other people fixing a problem, but the solution is almost worst than the problem !
    - Especially, when it's Linus' [laughter]  Linus is perfect [laughter]

- Have you been flamed by Linus ?

  - YES.  YES.  YES.  YES.  YES
  - Being flamed for coding style
  - Flamed at 2007 kernel site regarding scalability for device probing ...
    - "Stupid SGI people with their stupid problems, should stop bothering other people with their problems"
  - Broke something for a Mac Mini ... and Linus has a Mac Mini !
  - Linus flamed someone (and everyone involved in Linux security) as "wanking around",
      which means he really disagrees with you
    - This one was picked up on by all the technical press, e.g. article in The Inquirer
    - Linus is really quoted properly in context

- Question regarding security
  - Lots more code being committed these days, but not more code review
  - Some third-party organisations now have financial incentive to find security problems
    - Leads to more reports
  - Kernel hackers don't have the same types of incentives to find these sort of problems
  - Again: Just lots more code these days
  - Can't stop bugs getting into the kernel without slowing down the rate of development ... so ...
    Need some tool that will allow you to patch them out in the field as they are discovered [laughter]

Since I'm developing a mobile phone application, I need to keep up with this ...

Useful ... The Great Big, Constantly Updated, Australian iPhone FAQ v2.0

Sad, but no great surprise ... LEAKED: Telstra iPhone Plans -- and they'll break the bank

Good for us hackers ... iPhone OS 2.0 Unlocked :)

Clearly, the blog-o-sphere is going to be drowning in (infinitely recycled) iPhone 2.0 news.

Caught up with Christos and Droo at the Gypsy Bar (highly recommended, especially their Chocolate Torte ... the Salt-and-Pepper Calamari was also very good) ...

First week working on a new project.  Saving 5 hours commuting each week is excellent, especially if I spend half of that time doing some much needed exercise.  Best of all, this project combines the social networking trend with communications within a business organization ... so, I'm continuing to surf the front of the technology wave.

Bringing my development environment up to the latest versions (Fedora 9, Eclipse 3.4 and so on and on and on) has been fairly painless, it's getting easier all the time, but quite time consuming ... because there are so many tools and libraries.  Just keeping up with software development these days is a full time effort, there is obviously a lot of change, particularly now that the move to network-based applications is well under way, but still not yet mature (just look at how many RIA AJAX / Flash framework choices there are).

Another bonus of working from home, is that a pair of magpies for several yearsnow, having been bringing their offspring (during the June to September breeding season) down to my front lawn for a free feed.  They spend all day, continuously hunting for food for their demanding baby birds.  It's an entertaining break for a few minutes from hacking ...