The technical goal of Squeak 3.11 development is to provide a tool chain to enable many contributors to add code to the base without bottlenecks. The validation of this goal being the production of a new 3.11 image built using this tool chain.
For a change to appear in a release image, the following things must happen, in rough order:
Currently, all of these steps involve the time of a release team member, and thus do not scale well. Also, the last one, discarding a change, is very hard in the current process (update streams)
We are streamlining the process of incorporating changes into the release. We are making two policy changes, and using four new tools.
The stable version will include changes that have undergone a period of testing and found stable.
The unstable version will include everything in the stable version, as well as some less tested changes.
We don't want the scarcity of release team members to be a bottleneck in getting changes into the unstable version and exposing them to testing from those who try the unstable version. The release team will still have to bless changes to put them in the stable version.
We have added to SqueakSource an emailing capability. When new versions of packages are uploaded, SqueakSource will send out an email notification to interested parties. Using this feature, we created a mailing list that receives commit notifications for all projects that are part of 3.11. The goal is to make code reviewing a bit more involuntary. So far, we have fixed about 5 bugs as a direct result of someone doing an involuntary code review just because it was in their inbox.
A place to put image building scripts, as tasks. Tasks define dependents and pre-requisites, and are not always required to be run in a specific order. Sake/Tasks is intended to continue the role of this installer wiki, beyond the bootstrap stage (i.e. once LPF Packages and Tasks are loaded). Scripts are managed in the image, rather than on the web, and the scripts are under version control with Monticello.
A set of tasks, again defining dependents and pre-requisites, for each loadable package. The entire definition set managed with Monticello. For more information see Packages.
A further set of tasks, under the control of a periodic scheduler.
The build server will have several distinct jobs:
People will add tasks to the script that generates the image from the previous version. Mostly, these will be a list of mantis bug fixes to be applied. Currently, this list is on this wiki as one-line installer scripts (such as "Installer fixBug: 6721"); it is being transitioned to Sake/Tasks, where it will be version controlled.
When the build control script changes, The build server will run it and build new versions of the following:
This will be done for both the stable and unstable scripts, and for derivative images thereof.
In addition to building the basic image, the build server will be responsible for packaging up the image, VM, and associated files into platform-specific formats (Windows installer, Mac .dmg, debian .deb package, .rpm, etc.)
The basic image includes a number of packages pre-installed. The builder will update these when it builds a new image. This serves two purposes:
The build server will take a previously built image, install some packages (none, the set selected for the "full" image, or a random selection), and run all tests in the image. It will produce for download the image it tested against and a script to debug the tests that failed
The open release policy will take away a bit of the burden on the release team to find changes, listen to suggestions from the community, and remember them. Under this policy, the process of discovering changes can be parallelized across the entire community.
We could also perhaps automate this somewhat. Installer can query mantis, so we may use this feature to write a task to collate potential fixes.
Most changes come from mantis. A user can add a short installer script to any bug report on mantis.
Release tasks can then indicate dependencies upon particular fixes as in the following example:
TasksReleaseAfterSqueak310 >> taskCleanDeprecated ^ self define: [ :task | task dependsOn: { '6866 Refs Stop CurrentProjectRefactoring Removal' }. task action: [ Preferences noviceModeSettingChanged. "remove any old morphic key bindings" Installer mc unload: '39Deprecated'. ] ].
Unit testing is done automatically by the build server; no need to do it manually anymore
Packaging is done automatically by the build server; no need to do it manually anymore
There are two kinds of feedback a release team wants:
The way to get the first type of feedback is to make sure it is really easy for people to try out the latest changes. The build server gives beta testers two easy options:
To get the second type of feedback, we must make it really easy to see the code added to the image. This is why we have set up a [http://lists.squeakfoundation.org/packages
mailing list] that shows the latest changes made to the packages and code in the latest release. This way, interested people can do semi-involuntary code reviews just by reading some of the emails that go to that list. Easier reviews = more reviews = fewer bugs.
Under the previous policy, everything in a release is unstable, up until the moment that someone sends an email to squeak-dev saying "Squeak version X is now released and is available for download from the usual locations", at which point all changes became "stable".
Under the new policy, this is much more evolutionary; anyone is welcome to use the latest changes, and can be reasonably confident of their stability. Marking a new version is still a stronger declaration of stability, but we have added another level of granularity.
Also, since the image is assembled from a script, it is now possible, for the first time in squeak's history, to take a buggy change out of the unstable version of squeak. This could be done to an extent with the update stream, but it would create a situation where two squeak versions with the same update number had different changes, depending on whether they were generated before or after the deletion. Usually, one had to write another update that undid the previous update. Removing an update was flat out impossible in the 3.10 system; removing a change meant every change since then had to be re-made. This had to be done once, and caused the release team a great amount of pain.
This is, however, easy in this system. The night the change is removed, the build server will build a new version, with a new update number, which does not include the change. Users may or may not be able to upgrade an image to the new version, but that is acceptable.
Since the primary goal of 3.11 is the proving of the process. The base image itself will have no significant new features. Some tidying is planned together with the usual barrage of fixes, each of which will be fully documented in the image. In addition the inclusion of LevelPlayingField (and the ability to keep it updated) will provide the community with some significant benefits. Namely, speed, atomic loading, and some standard package types.
The Squeak 3.11 release will be released as '-basic' image. Contents of this image are as follows:
Having proved the tool chain subsequent releases will be time boxed. Packaging 4-6 months worth of fixes at a time, halving current release cycles.