Windows 8 DevCamp Postmortem

Last fall [Sep. 2012], my company Skyline Technologies, put together a Windows 8 DevCamp for our associates.  The purpose of the camp was to accelerate our employees learning of the Windws 8 development stack, have loads of fun and produce some publication worthy apps to the Windows 8 Store.  I've been meaning to write up a postmortem on the whole experience for quite some time, what follows is a postmortem on the DevCamp as a whole AND some specifics about our particular application, and represents my personal opinions on things.  I haven't had a chance to crowd-source the analysis with the other product owners to see if they had similar experiences.

What went right

Participant Sourced Ideas

Prior to the DevCamp, the camp leads sent out a call for proposals to all participants who had signed up for the camp.  Participants were asked to submit 1 or more application ideas which would be voted on by the group, and the top 5 would be implemented as part of the DevCamp.  This was a great opportunity for participants to flex their design and leadership muscles, and most [if not all] of the people whose proposals won ended up as the product owner for their idea.  This was a great way for developers to get a chance to 'own' the end product, providing additional incentive to push through the final publication and delivery process.

Great People, Awesome Team

Skyline, as a rule, hires some of the best people I've ever had the pleasure of working with, hands down.  The group that I worked with to deliver our Windows 8 app, Game Manager - Soccer, was a great mix of seasoned XAML devs, fairly new recruits and seasoned devs with little XAML specific experience.  Nobody on the team had any significant real-world Windows Store [/Modern UI/Metro] development experience to speak of.  Over the two days of the dev camp, and the follow-on work to complete the app for publication we ended up bringing the entire team up to speed on Windows Store development and had a great team dynamic that kept things fun.  I heard similar stories from every other team at the camp, and everyone I've talked to thus far has expressed an interest in doing future camps.

Good Technology Split

It turns out that after the voting was in, and product owners had chosen their desired technology stack, we had an even split between C#/XAML and HTML5/JS.  I was a bit suprised, being a XAML-geek myself, that more teams hadn't chosen the XAML stack.  After going through the camp and talking with my colleagues, the 'HTML5 guys' were equally baffled that anyone would choose the XAML stack.  The even split in technologies meant that our DevCamp participants could really pick and choose a team that fit both an app they were interested in and a technology stack that they had a passion for.  Win-Win for all involved.

Design Work / Project Scope

The team did a great job whittling down and mercilessly cutting scope to be able to fit the v1 release into the 200-300 hr window that the DevCamp required.  Building out screen designs and the planning that went into the application prior to DevCamp weekend made a huge difference in our ability to set focus and successfully deliver the quality of work that came out of the initial weekend.  We had learned this lesson a bit after our last company GiveCamp, and the DevCamp helped solidify that vision/requirement for future camps.  The good thing about needing to cut scope was that it allowed us to focus on quality for the pieces we were able to include, and to enhance the user experience beyond what we could have done had we kept the initial scope intact.

Surface RT

Who doesn't love to get new hardware!  Skyline generously handed out new Surface RT devices to all participants in the DevCamp as a reward for the time they put in on the project.  Not only is this a cool device, but it also allows us to test the application on 'baseline' ARM hardware to ensure that we are meeting the performance requirements of the system.  Using the RT allowed our team to find a number of performance related bugs before we published, which means a better experience for our users, and better ratings for the app overall.

What went wrong

Project Scope

Surprise surprise, scoping is good and bad!  Scoping a project is always hard, and even with the ruthless cutting of project scope, we STILL underestimated the total effort required to deliver the application.  I think a large part of this was caused by some of the items I'll get into below, and was certainly effected by our relative inexperience in the Windows Store app development space, but future efforts will definitely be tempered by this lesson.  In the case of our application, we really targeted what I [as product owner] felt was a minimum viable product feature set, so treading that line will always be hard.  When building/planning your applications, always keep in mind whether your minimum requirements put your project 'out of scope' for an effort such as a DevCamp.

Follow Through

This point is a hard one to swallow, and I don't fault any of our team members for it because, in the end, 'real life' is far more important than 'hitting a date'.  What went wrong in this process is that the scope [covered above] required a significant effort 'post DevCamp' to push through to publication.  As a result, a small portion of the overall project team ended up completely a lion's share of the remaining work and bug identification/fixes.  That extended effort was 'part of agreement' when signing up for the project, but unforseens and real life milestones tend to get in the way of the project plan sometimes.  To be clear, THAT'S A GOOD THING, and one of the great things about working for a company like Skyline.  The work-life balance is extremely important to the company, and to all our team members, so when real-life steps in, the project takes a back seat.  Bad news for the project, good news [usually] for those involved.

Local Data Model

As part of scoping things down, we went with a local in-memory data model for our 'data access' approach.  This was, I think, necessary in the end due to time constraints and project resources, but it did introduce a 'cognitive disconnect' for many on the team whose day-to-day job is n-tier/service-based application development.  It also introduced a number of bottlenecks and data management issues that simply don't exist when you have a 'real' backend to talk to, and when your source of truth doesn't 'handle' identity mapping, concurrency checks, relationships, etc.  In the end, the 'simple' approach which avoided SQLlite or some other 'real' database backend, required a lot of not-so-simple code to ensure we were always maintaining references during serialization/deserialization, passing data around between ViewModels, etc.  To add to this, our application supported data synchronization through the built-in WinRT 'roaming' folder from the start... until we discovered that the current defaults within WinRT limit that storage to 100KB of data.  As a result, we had to eliminate a 'free' feature and replace it with a far more complicated [and error prone] LiveSDK/SkyDrive approach [which we are planning to deliver in our first post-v1 release].

Testing / Feedback

Developing 'commercial' software requires an entirely new level of testing/QA/etc compared with Line-of-Business applications.  Now, to be clear, I think LOB apps could benefit from the robust level of testing and validation that should go into commercial software development, but the reality is that most LOB applications 'feel' more flexible in this department from the business/financial side of the house.  We grossly underestimated the effort required to adequately test the application, and the number of users required to do so effectively.  As a result, bug reports were late getting in and fixed and one or two people ended up doing a majority of the testing [which isn't always fun].  On future projects, having a dedicated 'beta' group that we can publish our application to ahead of time and gather feedback from will be on the top of the priority list.

SIDEBAR

As a brief aside, having a user-facing feedback site is also important.  We looked at [UserVoice](http://www.uservoice.com/) and [GetSatisfaction](https://getsatisfaction.com/), but both required fairly hefty fees for a non-single user account that we could use to gather feedback for the application [especially for a free app].  This is still something I want to do, and am pushing for, as it allows our user community to talk with us about their requirements/needs, something we don't always have during the initial application development. If anyone has any free or low-cost alternatives to these sites, I'd love to hear about it [leave a comment below].

Store Publication

The publication process was... interesting.  To be sure, it only took a couple attempts to get the application certified/published successfully, and I'm very thankful to the testers that helped us out and the team members that fixed bugs like mad, but the feedback from failed certifications needs serious work.  During certification, it was identified that our application experienced crashes, however, the certification results did not include any details about these failures, or what portions of the application were being tested when the failures occurred.  This makes the 'failure' result as close to useless as possible, and forced us to revisit the application and 'fix' things that weren't breaking locally [i.e. 'might' have been bugs] and really go out of our way to just try to guess what they might have tried that could have caused the error.  As an example, one of our team members discovered that if you put 'too much' text in our text boxes, the application would appear to freeze [and sometimes crash].  That's a great find, and certainly not something I would have thought to test for prior to this project, but if that was indeed what caused the crash during certification, it would have been great to have them tell us that so we could address it more directly.  Microsoft, if you're listening, recording a screencast of the testing/certification process that you can attach to failed certifications would go a LONG way towards producing better quality apps and shortening the turnaround time on publication failures.

Conclusion

Overall the DevCamp went better than I thought possible.  The camp was an awesome experience, both from a development and personal standpoint.  Our team did an amazing job and I think all the team members would [and should] be proud of what we were able to accomplish.  The lessons learned from this camp will be applied to the next one, and I look forward to future camps and continuing to make great software for Skyline and its many clients.  Thanks to everyone on the team, and at Skyline in general, for your support in making this camp and the applications that came from it happen!

... And ..., if you're a soccer coach [or know one], go out there and grab Game Manager - Soccer [or any of our other Windows 8 applications]!  Leave us some feedback so we can continue to improve and go out there and build your own Windows Store applications!