% Dpkg: The Interface
% Past, present and future. Defining traits, and effect on usage and evolution.
% Guillem Jover, 2015

# The Past

> - Origins
> - The dpkg Ethos

# Origins

- Debian announced on 1993-08.
- Changelog starts at 1994-08 (dpkg 0.93.5).
- dpkg-source first appears in 1996-08 (dpkg 1.3.0).

# The dpkg Ethos

> - Standard Unix tools
> - Metadata stored in deb822
> - Very thin plumbing

---

## Standard Unix tools

- (+) Recovery and portability are easy.
- (-) Not the most efficient/best ways to store data.

---

## Metadata stored in deb822

- (+) Can be repaired or edited with a text editor.
- (+) Requires a single style parser for most of the data in the archive.
- (+) Allows for solutions or workarounds that would not be possible otherwise.
- (-) Invites people to mess with those, even when they are private interfaces.
- (-) The deb822 format is not expressive enough.

---

## Very thin plumbing

- (+) Encourages easy exploration and trying out new things.
- (+) Gives freedom to maintainers to work in the way they are more
  comfortable or efficient with.
- (-) Non-standard and heterogeneous packaging, harder to work with
  from a global/QA PoV, and harder to get into for newcomers.

---

# The Present

> - Glue of an Ecosystem
> - Maintaining dpkg

# Glue of an Ecosystem

> - Very vast interface surface
> - Touches very disparate areas
> - Usage in Debian and derivatives
> - Reaches far beyond Debian and derivatives

---

## Very vast interface surface

- .deb/.dsc/.changes file formats
- DEBIAN/\* file formats
- run-time behavior guarantees
- command-line interface (not used only by people, also maintscripts, other)
- perl Dpkg module interface (public and private modules)
- libdpkg C interface (unstable)
- internal interfaces (e.g. dpkg database)

---

## Touches very disparate areas

- Build systems, dynamic/static linking, cross-building.
- New ports and architectures.
- New archive features.
- Binary and source packages.
- Policy.
- …

---

## Usage in Debian and derivatives

- Used by packages, written in any language or build system.
- Used by maintainers when packaging stuff.
- Used by sysadmins / users when handling installs/upgrades/removals.

---

## Reaches far beyond Debian and derivatives

- Debian is another downstream.
- Generality and portability are very important aspects.

---

# Maintaining dpkg

> - Decision boundaries
> - Changes can and will break something
> - Transition planning
> - Changes are traumatic
> - Easy to get into the hot spot

---

## Decision boundaries

- Pure dpkg implementation details.
- Cross-team responsibility.
- Debian-wide decisions.

---

## Changes can and will break something

- Many times it's obvious, and transitions can be planned.
- Sometimes it exposes broken stuff in few places.
- Sometimes it ends up being a matter of damage control.
- Sometimes the changes, even if legit, need to be reverted.

---

## Transition planning

- Announcement of intention to d-d(-a) or focused lists.
- Adding checks to lintian or other dist-wide checkers.
- Mass rebuilds to check for breakage.
- Emission of warnings in dpkg itself.

---

## Changes are traumatic

- The test suites have substantially ameliorated this.
- Temporary solutions have a high cost.
- Most time taken by design and thinking than coding.
- Probably the project where I'm most conservative.
- The Source-Version substvar case.

---

![Source-Version substvar deprecation](dpkg-substvar.svg)

---

## Easy to get into the hot spot

- Many times people demand conflicting features.
- People demand things that are very distro-specific.
- People demand workarounds/hacks.
- People demand insane things.
- People have said that dpkg development is both too fast and too slow…
- And many times, you sadly just have to say *No*.
- … As dpkg *is* The Interface, it is a focal point of conflict.

---

# The Future

> - Biggest deficiencies
> - Open problems

---

## Biggest deficiencies

- No file metadata tracking.
- No conffiles content tracking.
- No built-in binary package signatures.
- Lack of more declarative interfaces.

---

## Open problems

* Storing conffiles in a database.
* Storing file metadata.
* Making packaging easier.
* Source distribution.
* How to deal with the applications & containers fad.

---

# Questions?