polars-bio 0.36 reads protein structures and sequence alignments as DataFrames

polars-bio 0.36 reads protein structures and multiple sequence alignments into the same Polars frames you use for VCF and BAM. It also makes cloud reads parallel by default, opens the full 1000 Genomes PLINK 2 panel, and writes VCF headers back the way they came in.

Slide text

polars-bio 0.36.0 · 2026-09-21

polars-bio 0.36 reads protein structures and sequence alignments as DataFrames

Cloud reads also run in parallel by default, the 1000 Genomes PLINK 2 panel opens, and VCF headers survive a round trip.

The new structure readers take PDB or mmCIF files, a list or a local glob, and return a row per atom. Ask for residue level, and each row carries backbone coordinates and six angles, phi, psi and omega among them, in Angstroms and degrees. Foldcomp databases work too, decoding only the entries you select. Parsing runs in Rust, with no Gemmi or Biopython install.

atoms = pb.scan_structures(
    ["experimental.pdb", "predictions/*.cif.gz"])
residues = pb.scan_mmcif("protein.cif", level="residue")
residues.select("chain_id", "auth_seq_id",
                "phi_deg", "psi_deg", "omega_deg")
subset = pb.scan_foldcomp("afdb",
    ids=["protein_1", "protein_2"], level="residue")

A2M and A3M files load with the FASTA schema, each sequence exactly as written, inserts and gaps included. Stockholm gives one row per sequence per alignment, and a local, uncompressed Pfam seed file splits across partitions. For OpenProteinSet files with semicolon comments, pass a comment prefix. The readers are parity-tested against Easel, hh-suite and Biopython.

The concurrent fetches setting now defaults to 8 in more than 40 read, scan, describe and register functions. Whole-file reads from S3, Google Cloud Storage and HTTP become parallel ranged requests. On high-latency links, the old single connection took about twice as long as aws s3 cp. With 8 megabyte chunks, up to 64 megabytes can be in flight per stream. Set it to 1 to turn parallel fetching off.

In 0.35.1, the PGEN readers rejected the published 1000 Genomes fileset, because its compressed variant file passed a fixed 512 megabyte cap. Companion files are now streamed into a columnar table. That panel, about 75 million variants, opens in about 4 seconds within about 4 gigabytes, where the old layout would have needed about 45. Three new arguments set the limits.

A scan and sink round trip used to rebuild the VCF header, losing the file date, the source, bcftools provenance and the PASS filter. For a local file, the header now goes back line for line. Turn on preserve record layout, and each record keeps its own INFO and FORMAT key order. And an INFO_AF column is written as the input's AF, so an annotation tool's AF stays out of it.

lf = pb.scan_vcf("variants.vcf.gz",
    preserve_record_layout=True)
pb.sink_vcf(lf.filter(pl.col("qual") > 30),
    "filtered.vcf")

If you compare outputs across versions, check these. Reads are parallel by default. Written VCF puts GT first in FORMAT, as the specification requires. A malformed BED record now raises an error where it used to disappear. And polars-config-meta 0.3.2 is required, because older versions dropped the source header on collect.

To upgrade, run pip install polars-bio. Linux arm64 now gets a pre-built wheel too. This release does not write the alignment formats, an A2M or A3M scan parses in one partition, Foldcomp needs a local path, and a VCF from cloud storage still has its header rebuilt. polars-bio 0.36 reads protein structures and sequence alignments as DataFrames.

Slide text

polars-bio 0.36.0 · upgrade

$ pip install polars-bio

Linux arm64 now gets a pre-built wheel, built and tested on native ARM runners.

Not covered yet

Writing A2M, A3M or Stockholm files

Parallel parsing of an A2M or A3M scan

Foldcomp from remote storage (local paths only)

Verbatim headers for VCF read from cloud storage

biodatageeks.org/polars-bio