Things to do for piuparts
=========================


for 0.38 and later (split this list):

- move logfiles from bugged to archived if a newer version exists in pass
- move logfiles from fail to bugged if a lower version exists in bugged

- So it seems piuparts needs to get fixed to take into account packages
  being removed due to Conflicts. And probably should purge those
  packages which got removed, or reinstall them at the end of the run.

- svn mv piatti trunk/doc/examples  and include it into a new binary package. 
  when doing so keep in mind the two tango icons and credit those in 
  debian/copyright.

- PackagesDB: get rid of 50% of unique()

- remove configfile option "mirror" and use the mirror from "packages-url".

- piuparts-slave: if chroot.tgz is older than N days, regenerate it
 - currently this is done via ~piupartss/crontab, but that's a hack
 - only throw away tarballs (and recreate them) after testing that they can be recreated 

- fix #526046: introduce a whitelist of packages where it it is ok to break circular dependencies, like perl + perl-modules and xserver-xorg-core + xserver-xorg 

- master:
 - write stats about number of daily tests (-> report: create fancy graphs)

- untestable packages are not recognicable as such, see http://piuparts.debian.org/source/a/apache2.html 
  atm. (log is N/A, even though its there, even if empty) - http://piuparts.debian.org/sid/state-cannot-be-tested.html
  aint helpful neither.

- report:
 - write stats about the reasons for failures:
  - installation hangs and uses all cpu when installing with 
    DEBIAN_FRONTEND="noninteractive"
  - owned and unowned files after purge (policy 6.8 + 10.7.3)
  - files owned by two or more packages are not removed by dpkg, these should be ignored.
    eg, "/etc/php5	 owned by: php5-cgi, php5-common" from http://piuparts.debian.org/squeeze/fail/php5-cgi_5.2.6.dfsg.1-3.log
  - prompts without using debconf (#206684 is pending, will be in policy 3.8.2)
  - left-over processes (10.7.3)
  - "E: Broken packages" - often indicates a problem with the archive at the
    time of testing, not with the package being tested
  - "/var/lib/dpkg/info/.*.postinst: .* command not found" - missing depends
  - "/var/lib/dpkg/info/.*.postrm: .* command not found" - missing depends or
    using non-essential packages in purge
  - unclassified
 - include bugged status for packages on http://piuparts.debian.org/sid/state-failed-testing.html
 - RSS feeds of logs
 - include pts state (as an icon) in source summary pages
 - packages.txt for binary packages (like sources.txt)
 - do more fancy R graphs, eg. also per state
 - sort packages on maintainer pages alphabetically
 - link (and target) to piuparts.d.o configuration is static to piatti. should refer to the actual hosts configuration, eg goldwassers

- expire old logs a month after testing if newer logs exists (except if the new log is fail
  and the old is bugged, then the fail log should be moved to bugged first)

- create emacspeak-broken-dpkg-preconfigure package for broken repo. (then later
  put more broken packages in there and use that for testing piuparts)

- monitor:
 - write slave-watcher to monitor hanging processes, eg. looping dpkg-preconfigure
 - Check for and kill extraneous processes afterwards. Perhaps by checking
   whether their working directory is in the chroot.
   Introduce a whitelist of processes to wait for and assume it's an error 
   if those haven't been killed after $time=10min. see #387428
 - master.log grows to fast and there is no mechanism to stop it

- slave_run: cleanup stale proc mountpoints

- split the failed state into several problem-type failed states

- the templates used by update-reports.py should be taken from /etc/piuparts/templates/
  and not be included in the python source

- write reportbug wrapper for mass bug filing (start simple, make it more sophisticated 
  later)

- publish FAI classes to setup piuparts.$fqdn automatically 

- a redirect of http://piuparts.d.o/foo to http://p.d.o/source/f/foo.html would be nice


for 0.39 and on:

- multi-arch:
 - piuparts-report should have a list of available arch and list packages 
   only available on untested archs in a new state "(depends-)not-available-on-tested-archs"
 - master should (per default) only schedule packages which are not available 
   on the master arch to slaves of different archs -> "schedule-evenly-to-slaves = no"

- piuparts can't currently test upgrades of required packages. (Because they
  cannot be removed, it assumes these are untestable, which is only true for
  removal tests...

- find_default_debian_mirrors: if parts[2] contains a / (think
  stable/updates for security.d.o), you can't ignore this, it will break
  later...

- support for extra-packages-url (for volatile, security, etc)

- not sure if it's a sensible thing to to, but provide a way to turn off debugging
  output for piuparts.py - see http://docs.python.org/library/logging.html
- commandline-switches for all programms 

- 539146: piuparts can't currently test postfix, since installing postfix removes
  exim and removing postfix would require re-installing exim, and that
  doesn't happen; there are other packages like that, ie. debconf-english

- deal with packages that need to replace higher priority packages
  (which then need to be put back when the package is removed)

- mounting /proc and perhaps others (usbfs, sysfs, /dev/pts, etc.) in
  the chroot might be a good idea because some packages might need this.
  Interestingly enough this currently seems to prevent start-stop-daemon
  from starting any daemons ;)

- binary packages which have been binNMUed are shown with the wrong version 
  number on the package pages. this is just a problem in reporting, not in 
  testing. PackagesDB.get_source_package() gives back the right value, but 
  piuparts-reports does not care.

- from the FAQ: Just by looking at the Packages and Sources files it's 
  impossible to distinguish unknown packages (because they haven't build yet
  are only available on certains archs) and udebs, so that's why the state is
  so ambiguous. Future versions of piuparts-report are supposed to alse parse 
  the Packages files in the debian-installer directories, making it possible
  to properly detect udebs.

Low priority stuff (a.k.a. "nobody's said they must have it now")
-----------------------------------------------------------------

* move shell cronjobs functionality into master, slave & report

* make it possible to call aptitude (or similar) instead of apt-get and 
  allow to override the commandline arguments of the used program (to be 
  able to test with and without recommended packages or authentication).

* Remove --minimize and related options/functionality? Since 0.38 --skip-minimize 
  and --debfoster-options have become quite obsolete (--minimize has been added 
  in 0.38) since debootstrap now creates a minimal chroot.

* Kill children after a timeout to make sure the test doesn't run forever.

* #522913: include a test to see which packages which modify their own conffiles 
  so the user is presented with a dpkg conffile changed dialogue during
  the upgrade.  It would also be very interesting to see how many packages
  leave behind orphaned conffiles after purging a newer version which does
  not contain the files anymore.

* Bill Allombert: Does piupart test whether packages trigger useless
  conffiles handling ? (i.e. dpkg pretend you have modified a conffile
  when it has not) this is one kind of bugs I would really get rid of.
  h01ger: #466118 sounds like it does.

* piuparts-slave: keep track of reservations permanently
* piuparts-master: keep track of whom a reservation is made
* piuparts-slave: timestamps to log messages
* piuparts: upgrade the chroot before taking a snapshot of its files
