Reformat standard input to any width with #Python

#!/usr/bin/python3
import sys
import textwrap
w=int(sys.argv[1])
for line in sys.stdin:
print(textwrap.fill(line,width=w))

#Python script to show top 20 #bash commands:

#!/usr/bin/env python3
from collections import Counter
import matplotlib.pyplot as plt
import os
history_file = os.path.expanduser("~/.bash_history")
counter = Counter()
with open(history_file, "r", errors="ignore") as f:
for line in f:
line = line.strip()
if not line or line.startswith("#"):
continue
cmdline = line.split("|", 1)[0].strip()
parts = cmdline.split()
if not parts:
continue
cmd = parts[1] if parts[0] == "sudo" and len(parts) > 1 else parts[0]
counter[cmd] += 1
top = counter.most_common(20)
commands, counts = zip(*top)
commands = commands[::-1]
counts = counts[::-1]
plt.figure(figsize=(10, 6))
plt.barh(commands, counts)
plt.title("Top 20 Bash Commands (frequency)")
plt.xlabel("Usage count")
plt.tight_layout()
plt.show()

I programmed python for a living in the late 1990's, as I got into data analysis I switched to R in the very late 1990's and early 2000's, then in 2019 or so switched almost entirely to Julia. I also have experience in C, C++, Common Lisp and lots of other languages.

I need to write *modern* python for micropython for my project with my kid. What's like the 3 page zine refresher on modern python I should read? #python #programming #micropython

So i managed to load micropython on to one of my ESP32 devices. I am not clear how I load python code onto the board though. The docs weren't super clear on this. Any hints?

#esp32 #micropython #python #microcontroller

#HIRING! Senior Engineer, #python based full stack but leaning to devops/infra. 70/30 would be my ideal split. Fully remote UK only, £63,000 non negotiable sadly, but ~20% pension on top. Closing date: WEDS NOON! So get in touch (thayer@team-prime.com) asap if this might be you.

Org: healthcare, data, ~30 person team. Zero AI currently in place, but some may be implemented very gently/ethically/thoughtfully over the next year, or not!

Would esp suit someone who prefers chill vs rocket ship.

I'm looking for work, please boost!

I'm a senior software engineer with 35 years of experience. I've worked across an unusually wide range of domains: mobile game backends, privacy-preserving data platforms, high-throughput COVID testing infrastructure, email and account systems, e-payment processing, job marketplace systems, and bioinformatics. I pick up new domains quickly and have a track record of doing it repeatedly. I understand how to turn business needs into engineering requirements.

I've worked remotely since the 1990s and can operate with minimal supervision. I don't need hand-holding to find the right problem to solve. Several of my most valued projects were self-directed: I identified the need, built the thing, and shipped it.

Some of the technologies I'm familiar with include: Python, Perl, TypeScript/JavaScript, Haskell, Go, C, Java. Postgres, MySQL, SQLite. Flask, SQLAlchemy. AWS (Lambda, S3, RDS, SQS, EC2). Docker, Git. Github and Gitlab.

I've also repeatedly picked up new languages and stacks as needed: Haskell for differential privacy research, TypeScript for a 24/7 AWS Lambda system, Flask for my most recent employer. I've become productive with new systems over and over, and I can do it quickly.

I'm also a published author (Higher-Order Perl, Morgan Kaufmann), longtime blogger, and conference speaker with a reputation for making complex ideas clear.

My résumé is at https://plover.com/~mjd/cv/Mark%20Jason%20Dominus.pdf

mjd@pobox.com

Thanks for your attention!

#OpenToWork #remoteWork #softwareEngineering #Python #backend #hiring

#fediHire #getFediHired

The serialization of my book, Test-Driven Data Analysis, has reached chapter 4, which is available online at

https://book.tdda.info/book/chapter4.html

There is nothing more important than looking at data, and this chapter discusses how to construct a good combined profile and audit for each field in a dataset. Profiling shows the shape of the data, and auditing shows gaps, outliers etc.

Profiles for every field in a few datasets are also available in various forms at

https://book.tdda.info/profiles/

#tdda #book #books #testing #data #dataanalysis #visualization #python #rlang

Alt...

Left: A sample profile and audit for the CompanyName field from the Companies House Data. It has stats on null, whitespace-only and empty strings, then on most common names, then a frequency distribution for name length. Underneath are sample strings showing long, short, first, last, min, max and random names. Right: A profile and audit of the Price Delta field, which is real-valued, in pounds. It starts with counts for positive, zero and negative values and mean, median and mode. It then has a horizontal bar graph and table with a coarse binning of values, followed by a finer bar graph with broken bars. There is then a table of percentiles and finally a table of sample values (first/last, random, minima, maxima).

If you're within striking distance of Edinburgh, I'm speaking at @EdinbR tonight, after Mayuko Morgan: The thrifty data chef’s guide to cook up insight on a shoestring.

Free tickets available at: https://luma.com/vv57xyii

Do come along!

#data #datascience #documents #reprodicibility #rlang #python #book #books

Alt...

Title Slide: TEST-DRIVEN DATA ANALYSIS (TDDA) & TEST-DRIVEN DOCUMENT DEVELOPMENT (TDDD) EdinbR 2026–06-11T18:00:00+01:00 2.55 Bayes Centre, Edinburgh Nicholas J. Radcliffe Stochastic Solutions Limited & Department of Mathematics, University of Edinburgh

Alt...

The make process for a TeX/LaTeX document showing the various sources (.tex files, images, source code, outputs, definitions) being fed into the `make` command. Also in the process are tests of the included analytical outputs. If the tests fail, the book fails to build. The tests are generated by tdda gentest in this case.

Since there has been a huge influx of new users, I decided to write a new #Introduction and actually pin it to my profile.
I'm pushing 50 years old and I live in a Red State that is trying to make me illegal. I'm a #pansexual / #bisexual #transgender woman married to a heterosexual cisgender woman who frequently talks about the current hellscape for people like me in my Toots.
I'm #NeuroDivergent / #ND which is probably why all of these sentences start with "I".
I've worked in #InfoSec for a little over 20 years. I've had lots of roles in #SecEng, #SecOps, and #ThreatManagement. I taught myself #Perl, #Bash, #SQL, and #PowerShell. I'm decent at #JavaScript. I can read #Python and #Ruby. I enjoy automating things and turning manual processes into scripts.
I've been the primary #CareGiver to my wife for 8 years since she developed a chronic condition and went on disability.
My hobbies including #writing #paranormal short fiction, journaling my #dreams, and playing #PCGames on my laptop and #SteamDeck.
I prefer #StarGate over #StarTrek over #StarWars. Still waiting for Amazon to do something, anything with the Stargate property.
While we loved the #ArrowVerse including #Stargirl and #SwampThing, in general we prefer #Marvel over #DC.
I'm a fan of #Horror / #HorrorFam, #HorrorMovies and #HorrorBooks, especially the existential dread of #CosmicHorror or #LovecraftianHorror. I tend to sympathize with the nameless terrors. I am not a fan of mindless slashers, unrelenting gore, or torture porn. Over-the-top, egregious gore that crosses into the absurd is fine, though, so I am a Sam Raimi fan, obvs. Also, #HorrorComedies are underappreciated.
I'm slowly reconnecting with my #Pagan roots. I knew some stuff about #Tarot and had a friend who as a tree a lifetime ago and I'm trying to rekindle that.
We've got #Cats and they are our kids. I also happen to love #Frogs, but we don't have any of those.

#BLM #BlackLivesMatter
#TransgenderRightsAreHumanRights
#LGBT #LGBTQ #LGBTQIA
#ThePandemicIsNotOver
#ClimateChangeIsReal
#SexWorkIsWork

aw ye whats up my bozos it's static type checking for array shape and dtype constraints so you no longer have to toil in the mines of infinity incomprehensible (ndarray)->ndarray functions like bunch of clowns face smacking in a field of rakes

https://numpydantic.readthedocs.io/en/latest/typecheckers.html

#python

Alt...

E.g. say you have some analysis function that only accepts grayscale images: [python code follows. a type for 2D grayscale images and 3D RGB images are declared. then functions that return RGB and grayscale images, and finally a functino that only accepts grayscale image. demonstrates that the function fails a type chck when called with an RGB image] import numpy as np from numpydantic import NDArray, Shape GRAYSCALE = NDArray[Shape["* x, * y"], np.uint8] RGB = NDArray[Shape["* x, * y, 3 rgb"], np.uint8] def read_rgb() -> RGB: return np.ones((1920, 1080, 3), dtype=np.uint8) def read_grayscale() -> GRAYSCALE: return np.ones((1920, 1080), dtype=np.uint8) def grayscale_mask(frame: GRAYSCALE) -> GRAYSCALE: # Probably something fancier than this... mask = np.zeros((frame.shape[0], frame.shape[1]), np.uint8) mask[frame > 5] = 1 return mask # this works grayscale_mask(read_grayscale()) # this doesn't grayscale_mask(read_rgb()) examples/incorrect/rgb_gray_frame.py:28: error: Argument 1 to "grayscale_mask" has incompatible type "ndarray[tuple[int, int, Literal[3]], dtype[unsignedinteger[_8Bit]]]"; expected "ndarray[tuple[int, int], dtype[unsignedinteger[_8Bit]]]" [arg-type] grayscale_mask(read_rgb()) ^~~~~~~~~~ Found 1 error in 1 file (checked 1 source file)

Alt...

Shape checking Scalars Scalar shapes are checked as expected, where the shapes must match exactly. [python code follows. a correct example shows an NDArray annotation with a Shape[1, 2, 3] (three dimensions with sizes 1, 2, and 3 respectively) as a return value coming from a np.ones() constructor and assigned to x as passing a type check. an incorrect example shows that the type checker fails both between the numpy constructor and return type and the return type and the assignment] Correct def make_array() -> NDArray[Shape[1, 2, 3], np.uint8]: return np.ones((1, 2, 3), dtype=np.uint8) x: NDArray[Shape[1, 2, 3], np.uint8] = make_array() Incorrect def make_array() -> NDArray[Shape[2, 3, 4], np.uint8]: return np.ones((1, 2, 3), dtype=np.uint8) x: NDArray[Shape[5, 6, 7], np.uint8] = make_array() error: Incompatible return value type (got "ndarray[int, dtype[_8Bit]]", expected "ndarray[tuple[Literal[2], Literal[3], Literal[4]], dtype[_8Bit]]") [return-value] return np.ones((1, 2, 3), dtype=np.uint8) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: Incompatible types in assignment (expression has type "ndarray[tuple[Literal[2], Literal[3], Literal[4]], dtype[8Bit]]", variable has type "ndarray[tuple[Literal[5], Literal[6], Literal[7]], dtype[_8Bit]]") [assignment] x: NDArray[Shape[5, 6, 7], np.uint8] = make_array() ^~~~~~~~~~~~ Found 2 errors in 1 file (checked 1 source file)

Alt...

So, if enabled, return values for non-numpy interfaces can declare how to infer their shapes and dtypes: [python code follows, demonstrates that numpy and zarr constructors are correctly typed from the sizes and dtypes in their constructors (dask is broken, note at bottom). without the plugin they are just generic arrays and Any types] from typing import reveal_type import dask.array as da import numpy as np import zarr x = np.zeros((3, 4, 5), dtype=np.uint8) y = da.zeros((3, 4, 5), dtype=np.uint8) z = zarr.zeros((3, 4, 5), another=int, dtype=np.uint8) reveal_type(x) reveal_type(y) reveal_type(z) Without the plugin note: Revealed type is "numpy.ndarray[tuple[int, int, int], numpy.dtype[numpy.unsignedinteger[numpy._typing._nbit_base._8Bit]]]" note: Revealed type is "Any" note: Revealed type is "Any" With the plugin note: Revealed type is "numpy.ndarray[tuple[Literal[3], Literal[4], Literal[5], fallback=int], numpy.dtype[numpy.unsignedinteger[numpy._typing._nbit_base._8Bit]]]" note: Revealed type is "Any" note: Revealed type is "numpy.ndarray[tuple[Literal[3], Literal[4], Literal[5], fallback=int], numpy.dtype[numpy.unsignedinteger[numpy._typing._nbit_base._8Bit]]]" Dask is broken Note that dask’s constructor inference doesn’t work at the moment. This is due to dask’s array creation routines being positively haunted, an untyped wrapped dynamic construction of a function that creates a class that creates a class. PRs welcome re: figuring out how to type that.

We're cookin', y'all.
I'll be done soon, but just you wait, it's getting brown on the edges.

#sigal #python #caddy #selfhosted

Alt...

testing out sigal for a gallery