FUDCon Toronto Trip Report

FUDCon Toronto 2009 is over and I’m on the way home.

I Attended some good barcamp sessions:

• Cloud Computing + Fedora and Amazon EC2
• Fedora, Zikula and Fedora Insight
• Can’t we all just get along? – Sysadmin & Developer Panel
• Designing UI mockups in Inkscape
• MediaWiki syntax for non-experts

My favorite session was the Inkscape session by Máirín Duffy (aka “Mo”).  It was packed with information and gave me just the right level of detail to get started with Inkscape (an open source replacement for Adobe Illustrator).  I was thinking of asking her to do a session like this, but she already had it planned.

I also worked on several wiki pages to help better record our existing release processes.

The biggest monster of them all was finalizing the Release Criteria pages.  I had been playing around with different ideas and drafts since the beginning of Fedora 12.  It was really great to launch the pages publicly a few weeks back, collect feedback and tune them for use in Fedora 13.  They have changed a lot since my first ideas, but what matters most to me is that we have a larger framework now and support for doing it.  Thanks to Bill Nottingham, Tim Burke, James Laska, and Adam Williamson for suffering through all the details and making the pages much better during most of the first hackfest day.  And another big thanks to Adam for circulating the pages to lots of other people and mailing lists for feedback.

For the past few months I’ve also been doodling on some ways to better document our release processes–how to complete specific tasks and how we make decisions.  This is important for running smoother releases and getting more people involved.  So far it has been difficult to know where to start or how to represent things.  Part of what helped me get started was the posts I did on mind mapping.  The release criteria came out of the same effort.  A few weeks ago I was excited to watch a Bugzappers meeting where they followed the housekeeping SOP I painstakingly created a couple of releases ago for creating tracker bugs.  This is a victory because now the process has scaled beyond just me.  The same thing could work in Release Engineering.

I went through all of the release engineering tickets for Fedora 12 to create a starting list of all the tickets that need to be created during a full release cycle. I reviewed and tuned the list with Jesse Keating and am also requesting feedback on the release engineering list. This has all the makings of some great ”wiki-fication” by linking to more detailed pages explaining each task combined with automating the ticket creation process.

During Fedora 12 I found that there was no clear documentation explaining how we decided when we are “done” and the release is “ready.” The new release criteria pages talk to the “done” part.  A new “Go/No-Go” meeting SOP explains the process to bless a public release (Alpha, Beta, or Final) as “Gold.”  James and Jesse looked over the first version.  Next I’ll be sending it out to the lists to get more feedback.

In the scheduling department, I met with Marketing, Quality Assurance, and Release Engineering to perform a detail review of the team schedules I’ve drafted for Fedora 13.  Mel, James, and Jesse gave me lots of feedback and changes for the next revision.

I also had a lot of hall way conversations with people about different aspects of Fedora and things I’m helping to move forward in Fedora.  Many of the things I got done were made easier by having so many people from different places in the same room at the same time.

At past FUDCons I’ve usually left during part of the last day. It was fun to go back to the hotel, have a little down time, and meet up again with people at the hotel for “Hack and Snack.”  Best of all Mo took some pictures for me and created my first ever (and very fantastic) hackergotchi.  I can only dream of being able to use the gimp (an open source replacement to Adobe Photoshop) as effectively some day.  Thanks again Mo!

Marketing to Fedora’s Target Audience

I am thinking of Fedora in light of an article about Billy Mays on Copy Bloggler.

“So how do you know if the product you’re currently selling or developing is great… and easy to sell? According to Billy Mays your product must have these 5 Essential Character Traits:”

1. Solve a problem
2. Have mass appeal
3. Be unique
4. Offer instant gratification
5. Be demonstrable

The Fedora Project is not selling anything.  It is all free, but we still want to create a solid product that is wildly successful.

But great salesmanship, contrary to popular opinion, is not about selling ice to Eskimos.

The truth is less flamboyant, and far more reasonable.

Simply put, behind every great salesman is a great product. And Billy Mays understood that better than most.

Because if it’s a great product—it was easy for Billy to sell, using salesmanship techniques he had honed over two decades of selling.

As the Fedora Project continues to define its Target Audience these are great points to think of in terms of how we attract people to the Fedora Project and what it looks like to create a compelling Linux distribution.

Some people people may chafe at the notion of the Fedora Distribution as a “product.”  I can understand that, particularly if working on a “product” is associated with strict process, lots of bureaucracy, endless meetings, and the pressure to constantly generate more revenue.  Maybe that is one reason some people think defining a target audience for the Fedora distribution is going too far.

Here is how I see these five points applying to Fedora.

Solve a problem–Many people will try Fedora for fun, but if it really solves a problem for them they will stick around.  They will also share it with others.

Have mass appeal–Fedora should put its best foot forward among the segment of people that we identify Fedora for. This point may not fit Fedora as well.  We maybe not be able to appeal to as many people as a stain remover.  The heart of the “Target Audience” discussion is that Fedora cannot be everything to everyone.

Be unique–A lot of Linux distributions look the same.  A lot of open source projects have the same goals and structure.  What really differentiates Fedora and how can we call that out in a way that corresponds to these other attributes?  We may need to differentiate ourselves more than just being the “free-est free” Linux distribution.  Especially if our Target Audience does not understand or place a high value on the pureness of Fedora’s free-ness.

Offer instant gratification– If Fedora “just works” people are less likely to abandon it in favor of something else.  If it solves a problem they have immediately, in a meaningful way, why would they go somewhere else?

Demonstrable–if we can’t clearly and easily demonstrate that the Fedora Distribution: solves a problem, is appealing, is unique, and gives instant gratification, new users may go somewhere else.

Admittedly, Fedora may not fit in as broad a market as OxiClean, so I’m thinking of these points in the context of our identified target audience.  Some of these are a stretch, but they are great questions to ask ourselves if we want Fedora to be great.

Addicted to Opining

Chris Brogan raises some interesting thoughts and questions in his post “Are We Addicted to Giving Our Own Opinions“

The tools we use for social media have empowered us to be steady-flow commentators. Watch Twitter or Facebook during any event, and you’ll see our added commentary rolling along in time with the experience…. In blog comments, on Twitter, all over Facebook, Yelp, YouTube, and several other sites, we’ve been groomed to give our opinion. We spit it out everywhere. We share, rate, criticize, deride, praise, and everything in between.

I’m not sure what to call it.  In some contexts it does appear to be an addiction and in others, a sense of entitlement.  The sense that, “You’ve said something in public and as a result, I’m entitled to give my opinion and you have an obligation to receive it.” I’m not against the idea of free speech, but I am uncomfortable with the notion that I have an obligation to receive everyone’s opinions.

Several months ago I disabled comments for a post on my blog.  I was tired of long email threads on the Fedora mailing lists where the experience was more a battle of who could have the last word and why the previous person’s opinion was wrong versus thoughtful dialog.  The “discussion” was less about thoughtfully considering unique points of view and more about “being right.”  My post was about a topic that had already been discussed on the Fedora lists.  I wasn’t looking for more ideas and feedback.

A few people were annoyed because they couldn’t leave their opinions at my blog.  To my knowledge I had not asked for their opinion. I simply wanted to share something from my own site without five people immediately telling me I was wrong. They were free to do that from their own blog using trackbacks, but some thought that was too inconvenient.  In some ways I can see that it was if they were approaching my blog with the expectation that it was a place for their opinions too.

I still haven’t made my mind for good on this.  Mostly I see my blog as my own little house on the internet where sometimes I invite visitors in and sometimes you get to read what’s posted on the front door. For now I’ve taken a more nuanced approach where I turn on comments for posts where I am looking for feedback and opinions.  When I just want to think out loud and give my point of view without having every bit of it critiqued I turn them off. Maybe this is the wrong approach.  Maybe there is a better middle ground I haven’t thought of yet.

Brogan concludes by asking,

So the question becomes: if we’ve built all these tools, these comment buttons, these like buttons, these “share and add notes” buttons, how is this impacting our interactions and our communication? Now that we’ve gone from not having a voice to having tools to give our opinion about everything, how does this change us? How does it impact how we interact with people? What does it mean to the larger ecosystem?

My first reaction is that none of this is helping us to be more thoughtful listeners–we aren’t doing ourselves a favor by re-enforcing a behavior that seeks first to talk and then to listen.  Perhaps you’ve seen this in a conversation.  One person shares something and without missing a beat another person jumps in with their own experience or story without explicitly acknowledging what the other person said. I’m not saying it always has to be that way.  Sometimes sharing your own experience is an implicit way of acknowledging and valuing what another person said–and sometimes it is more about taking the mic back for yourself.

I fully realize that the other side of this is that we can learn a lot from each other.  I’ve learned a lot from other people’s opinions and way of seeing the world.  And a lot of that has happened through comments and long discussion threads. What is most often missing for me is the sense of constructive discussion–where an idea becomes more robust, knowledge increases, and the conversation goes deeper.

Chris Grams wrote an insightful article about the phenomenon of “memo-list” at Red Hat.  Memo-list is capable of providing the same level of experience (the highs and the lows) of the Fedora mailing lists.  I think his conclusions are a good response to how to use our communication and commenting tools better.

The most important principle is that there are no stupid ideas– all ideas are good (in his post, Gary Hamel says “All ideas compete on equal footing”). If your goal is to use an open forum to get the best ideas, you must generate a lot of ideas. And if you want to get a lot of ideas, people must feel safe to contribute without fear of harsh criticism. If people begin to fear criticism, they will self-edit. No more openness. And a lot less ideas.

The second principle is that when you feel the urge to criticize an idea (hey, we all do…), resist, and instead come up with a better idea and rally people around it. Keep the conversation positive, constructive, and stay focused on creating rather than judging.

Maybe mailing lists are not “social media” in Brogan’s context, but they are definitely places where people give their opinions.  The reminder I took from Grams’ post was the importance of being clear about your goal.  I suppose social media can have a variety of purposes and some people will use the same medium for different goals.

In the spirit of getting everyone’s opinion I’m asking for yours.  Comments are open.  Tell me what I’m missing or should thoughtfully consider.

Our Fragile Existence

This Thursday is Thanksgiving, an official holiday in the the United States.  I am extra thankful this year.  It wasn’t so long ago that I felt more invincible.  Aging has a natural way of reducing that.  Potentially fatal accidents, more so.

Almost three weeks ago I took what I thought would be a short break from work to finish cleaning out the rain gutters at home.  With the rain gutters done, I carefully exited the roof placing both feet and my full weight on the first rung of the ladder below the roof line.  The ladder slipped out from underneath me and I fell to the concrete patio six or seven feet below.  Needless to say my plans for the rest of the week were instantly altered.

I could have fallen or landed on that unforgiving concrete patio in any number of ways.  At times I still have a hard time accepting how fortunate I am and how potentially fatal it could have been. I know, I should just be glad I’m mostly okay.  But my mind doesn’t work that way and the results of this event are too significant simply to dismiss.

I landed on my side with my hip taking the brunt of the fall.  Amazingly, it is only bruised.  X-rays show minor spinal damage which should eventually heal.  In the meantime I’ve had some excruciating back pain and muscle stiffness that makes it impossible to bend over or lift anything heavy.  Ibuprofen helps a lot and I have come to know the magic healing properties of ice–ten or fifteen minutes (no longer) of ice every hour does miracles.  I also have an entirely new level of appreciation for anyone with back pain.

Things I’ve learned and experienced from this incident:

• Things don’t always go as planned
• I can’t make logical sense of everything
• Next time tie the ladder off to a strong rain gutter bolt
• If a ladder moves when I’m not on it, have someone else make sure it is still safe from the bottom
• Other people need me more than I realize
• I need other people more than I realize
• Grace and divine intervention happen

Fedora Target Audience & Release Criteria

As the Fedora Board and larger community has discussed what we want our releases to be and who they are for–Target Audience–I’ve wondered more and more if a reworking of our Release Criteria could help make parts of our decision process clearer and easier to follow.  By clearly defining our Target Audience we should be able to write more specific and accurate release criteria. The recent PackageKit controversy highlights this importance.  The settings for this package made certain assumptions about a particular target audience and user of Fedora.  It makes me wonder how many other parts of our distribution make different or contradictory assumptions about our target audience–resulting in a distribution that is not as coherent and consistent as it could be.

A clear definition of our target audience might also reduce the emphasis we place on developing and testing certain areas, freeing up time to make other things better.

How do you know when testing is done?  How do you know when the product is ready to release?  Does the decision to release seem similar to deals done in the dark, smoke-filled rooms, or as if there are rules of the game that you don’t know?  Does the release decision sometimes seem completely arbitrary?  If you’re finding it difficult to make rational decisions about when to release the software, developing and using release criteria can help–Johanna Rothman

I’ve been more or less actively involved in Fedora since Fedora 7.  Even though I’ve just completed my fifth release, am actively involved in creating the schedules, and lead the readiness meetings before each public test and final release, this paragraph still resonates with me.  This has long been one of my frustrations of trying to understand and participate in Fedora development process.  Things have definitely gotten better over the past few releases.  We can still do a better job without adding layers of bureaucracy which some people fear.

To move this process forward I took a first shot at an enhanced release criteria framework for Fedora 13.  Ultimately I believe we need a stronger framework to be more successful, but I’m not saying the framework I’ve proposed is the one we have to use.  A lot of the information included has been in existence for a long time at the QA Release Criteria page.  From it I created three separate pages–one for each public release: Alpha, Beta, and Final as well as an introductory page.   I also added a few extra proposed bullets.  I copied and pasted most of them from Release Criteria and adapted the ‘’shoulds” and ”musts” as specified–where ”must” was required for all releases and ‘’should” was only required for the final release.

I think a separate page for each release is helpful because in reality each release has a separate target audience.  The audience and goals for the Alpha release are much different than the audience and goals for the Final release. Certainly there will be overlap in these audiences, but we sell ourselves short if we assume the audience and goals for all three releases are the same.  My hunch is that some of these requirements are dated and wrong which makes reviewing everything for Fedora 13 a good thing.  In light of the security discussion around PackgeKit we may also want to add a security aspect to our release criteria.

Another benefit of a clear list of release requirements is that the “Go/No-Go” meetings become more objective.  There is less of a need to subjectively take everyone’s pulse and ask people how they are feeling.  Instead we review the list of release requirements, which also encompasses blocker bugs.  If the requirements are met we ship.  If they are not met, we add a week the schedule and try again.  The definition of what makes a bug a release blocker becomes clearer as well.  In addition I’ve started a new blocker bug FAQ.

Check out the wiki pages and join in the discussion on fedora-test-list where I’ve started a thread on the same topic.  My hope is that we can refine and discuss these pages over the next two weeks and then meet at FUDCon in Toronto to discuss and put a framework in place for Fedora 13.

Fedora Talk Activity Day

The Fedora Talk “activity day” was two full days packed with good discussions, learning more about Asterisk, sharing some good laughs, and getting a lot done.  Paul Frields did a great job coordinating the logistics for the event including our lodging and work place.  Paul picked a great location called Business Playce, a co-working office in Fredericksburg, Virginia.  The owner, Paul Delagrange was a fantastic host.  After seeing all our gear and noting that a bigger room was available, he “upgraded” us to a larger conference room on the spot.  This turned out well considering the tangle of cables,  phone hardware, and power cords we had assembled in the first hour.  Jeff Collie brought a nice speaker phone which we connected to Fedora Talk so others could call in.

The main purpose of this event was to get Asterisk gurus Jared Smith and Jeff Collie in the same room with the rest of us–Jon Stanley, Paul Frields, and Ian Weller, to nail down and implement Fedora Talk’s ability to record and stream conferences.  We also spent a fair amount of time increasing the depth and quality of Fedora Talk’s documentation which will hopefully be published soon at http://talk.fedoraproject.org.  A reminder that key links are in the navigation panel on the left even though they look like the regular sidebar links from the wiki.

We had great participation from all over the world.  Not only did six of us gather in Fredericksburg, Virginia, but a number of people called in from: Chicago, Brisbane Australia, Washington State, Utah, and other unknown locations.  We were able to provide this conference, free of charge for sixteen hours using the existing Fedora Talk set-up. On the phone we had help from Clint Savage in the way of setting up the IceCast streaming server, Mike McGrath answering Infrastructure questions, and Darren VanBuren and Clint working on documentation and creating screen shots.  Bruno Wolff also helped with a variety of things including testing of our new documentation.  Others helped too so if I missed you and forgot to mention you did send me an email or chime in on the comments.

I focused on the overall facilitation of the event by helping to guide the sprints and keep us focused on meeting our goals.  Along the way I learned how to configure my new (to me) used Grandstream phone, format documents for the Fedora website, and increase my git knowledge–it was funny and sad how many types I attempted to issue a git command by starting with cvs.  I also spent a lot of time updating the use cases and making sure we they were all documented and linked properly.  We discussed the possibility of taking another run at the unfinished tickets and documentation at FUDCon in Toronto (December 5-7, 2009).

One of the most interesting things I learned at this event was that Fedora is blazing a trail in the area of providing an open framework for managing the recording and streaming conferences with Asterisk.  As were getting started I asked Jared Smith (who also happens to work at Digium and would know better than anyone else) what other existing solutions we could use instead of “writing all this new stuff ourselves.”  Jared noted that while a lot of people have developed applications to manage call recordings and streaming, to his knowledge all of those applications have been developed privately and non of them are open source or freely available.

Specific things we accomplished:

• Thorough list of use cases for three types of Fedora Talk roles: Callers, Conference Facilitators, and Administrators
• Documentation for each of the use cases or URLs to open tickets
• Ability to record and/or stream conferences
• Call recording notifications
• Protoyped web application to control recorded or streamed conferences
• Comprehensive documentation for configuring and troubleshooting soft phones
• Jared enabled numerous bits of Asterisk functionality on the spot as result of simple “Can Asterisk do…?” questions.

Things to consider in the future:

• Identify specific tasks in advance that people on the phone can help with
• Have one person on the phone take the lead for helping coordinate tasks with other people on the phone and IRC or forming pairs to work on tasks

Key success ingredients:

• Being very clear, reasonable, and focused from the start about what we wanted to accomplish
• Lots of advanced planning, including mailing list discussions and IRC meetings
• Detailed use cases created before the event.  Use cases with people’s names made them more personal and fun.
• Having subject matter experts present–not on the phone, but in person to draw diagrams and explain complicated technical things in person and in real time
• Working in short sprints (45 minutes to an hour) to avoid getting too bogged down on a particular topic or going down needless rat holes
• Giving status to the rest of the team at the end of each sprint to explain what we had worked on or accomplished and what we learned.
• Working in pairs with a small group (six seemed to be a really good number).
• Being flexible about how we used our time while maintaining the same goals.  The sprints worked well for the first part of the first day.  At other times they did not fit the nature of the tasks we were working on.
• Outside of defined sprints, periodic check-ins with the group about progress and what we wanted to work on next–no less than once every 1.5 hours.
• Having lunch delivered to us so we could all eat and keep working–this added 1.5 hours to the day that would have otherwise been lost by going out, coming back, and starting up again.

For me the biggest win was the use cases we created in advance.  They were a critical filter to run new ideas through and keep us focused on what we wanted to accomplish and how we wanted to define “success” for the event.

Thanks again to all the people who participated and gave their time!

More FedoraTalk And More Action

The long anticipated Fedora Talk Activity Day starts this Friday. If you have ideas or special needs for Fedora Talk, please update the use cases on the wiki.  See our overall plan for additional details.

We’ve been doing as much planning and organizing in advance as we can so that once we meet in person our time can be focused on implementing. I’ve been to several hackfests and FADs where a majority of the time was spent identifying and detailing things we need to fix in Fedora. While valuable, the implementation of those ideas usually takes several releases (a year or more) and some never quite get done. There are valid reasons for this I suppose, but having seen the pattern repeat itself I wanted this event to be different. My hope is that this FAD will be known for the things we got done and not so much the things we plan to do later.

For the past four weeks we have been planning and brainstorming on the infrastructure mailing list, creating wiki pages, and meeting on IRC. While not as fast as meeting in person, the overall results may be better because we’ve had time for our ideas and plans to ferment.  We got a lot of admin stuff out the way like figuring out what packages we need and what servers need to be rebuilt before we start.  Bruno Wolff also did a great write-up explaining how to setup Asterisk yourself.

One of the best ideas to come out of our planning meetings (thanks to Jeffery Ollie) was creating use cases to capture the needs of FedoraTalk users and administrators. These use cases form a natural link to open infrastructure tickets (RFEs) and documentation explaining how to execute each use case. By the end of the FAD we will have documentation for the use cases that are complete and open tickets for the use cases we intend to address in the future. This was another great discovery. A lot of great ideas from other Fedora events get lost in the sea of wiki pages, blog posts and and follow-up email discussions. Using this method, hopefully everything is in one place with a clear path to the documented implementation or a ticket reminding us we need to address it in the future.

I’m also hoping we can put to to use some things I recently learned about code sprinting at the Open Source Bridge conference–ironically the same place where the idea for the FedoraTalk FAD was born. I’m excited to see what kind of results we get from working in pairs and time boxing mini-sprints to 45 minutes.

Add The Key For Signed Rawhide

Somewhere along the line the packages in rawhide (soon to be Fedora 12) became signed.  This is a good thing.  I don’t run a current rawhide box all the time so I’m probably the last one to know, but tonight when I tried to perform an update it kept stopping because it couldn’t verify the signatures on each package.

warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 signature: NOKEY, key ID 57bbccba


A couple of educated guesses later and everything works.  This important page has a list of the keys for each release.

Import the public key for Fedora 12:

# rpm --import https://fedoraproject.org/static/57BBCCBA.txt


Installing or updating packages should work fine now.

The other alternative to all this is that I did something goofy way back when resulting in a strange yum repo setup

Resistance or Not

One of the things I love about working in the Fedora Project is the opportunity to blog.  In the beginning I thought blogging was for other people.  Then I realized I needed a place to talk about the things I was working on.  Then it became more than that.

There is something to this “writing thing” that resonates with me and makes me want to become a better writer.  Along the way I’ve read a few different books about writing.  Currently I’m reading Writing Down the Bones by Nataile Goldberg.  Like most of the good books I’ve found on creativity and writing, they were recommended by Merlin Mann (see the reading list at the bottom).

Recently, Goldberg was talking about resistance and all the excuses we can come up with for not writing.  Her suggestion was to spend a little bit of time writing those things down before getting on with your writing.  Another good book about resistance is The War of Art.

I resist writing because I tell myself that nobody will read what I write because there is such an ocean of other stuff out there written by other people–why would anybody read what I have to say?  Or I tell myself that the blog posts I have in process or would like to write about are too complicated and will take too long to complete.  And by the time they are completed Fedora will have moved on and those thoughts won’t be relevant any more.

Sometimes it isn’t so much about resistance as it is the thoughts and ruminations about other things going on at work or at home.  A million questions fly in and out of my head as I try to “just write.” My conclusion is that sometimes you just have to come back to it.  It cannot be forced.  Other times brute force is just what is needed.

How do you know when you genuinely need a break or when you are just trying to talk your way out of having to do what you should be doing?

Fedora 12 is Full Steam Ahead

This is a little bit of a “fluff” post to satisfy those readers that are hitting my blog based on the title of a post in April 2009 (before the Fedora 12 Schedule was even finalized) who maybe disappointed by what they find there.  Fedora 12 is still under development.  Its planned release date is November 17,2009.  If you want a sneak peak, the Fedora 12 Beta will be released on October 20, 2009.  It will be available at Fedora’s pre-release download location.

Here are some important Fedora 12 links you might be looking for:

• Feature List
• Macro Schedule
• Micro Schedules

Or maybe you are looking for some ways to help out.  There is never a shortage of things to do in Fedora.  Helping out can mean working on something that is not your area of technical expertise, but makes everyone else’s lives easier.  Just today I helped call out the bugs for the Fedora 12 Beta Blocker review meeting.  I announced the meeting and helped facilitate the note taking and bug updating, but I can honestly say that none of my technical knowledge (or lack thereof) was used.

If you want to get involved, there are lots of choices in a variety of areas.

On the other hand if you just want to download the latest version of Fedora, that would be Fedora 11.

If you don’t want to download Fedora, but plan to be at LinuxCon next week (September 21-23, 2009) in lovely Portland, Oregon–visit the Fedora Booth and we’ll hook you up there!  Even if you already have Fedora, stop by and say hello.