Python Bytes

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1: Polars: Lightning-fast DataFrame library for Rust and Python Suggested by a several listeners “Polars is a blazingly fast DataFrames library implemented in Rust using Apache Arrow Columnar Format as memory model. Lazy | eager execution Multi-threaded SIMD (Single Instruction/Multiple Data) Query optimization Powerful expression API Rust | Python | ...” Python API syntax set up to allow parallel and execution while sidestepping GIL issues, for both lazy and eager use cases. From the docs: Do not kill parallelization The syntax is very functional and pipeline-esque: import polars as pl q = ( pl.scan_csv("iris.csv") .filter(pl.col("sepal_length") > 5) .groupby("species") .agg(pl.all().sum()) ) df = q.collect() Polars User Guide is excellent and looks like it’s entirely

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Michael #1: auto-py-to-exe Converts .py to .exe using a simple graphical interface A good candidate to install via pipx For me, just point it at the top level app.py file and click go Can add icons, etc. Got a .app version and CLI version (I think

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1: The Python GIL: Past, Present, and Future Bary Warsaw and Paweł Polewicz Michael #2: Announcing the PyOxy Python Runner PyOxy is all of the following: An executable program used for running Python interpreters. A single file and highly portable (C)Python distribution. An alternative python driver providing more control over the interpreter than what python itself provides. A way to make some of PyOxidizer's technology more broadly available without using PyOxidizer. PyOxidizer is often used to generate binaries embedding a Python interpreter and a custom Python application. However, its configuration files support additional functionality, such as the ability to produce Windows MSI installers, macOS application bundles, and more. The pyoxy executable also embeds a copy of the Python standard library and import

Watch the live stream: Watch on YouTube About the show Sponsored: RedHat: Compiler Podcast Special guests Mark Little Ben Cosby Michael #1: libgravatar A library that provides a Python 3 interface to the Gravatar APIs. If you have users and want to show some sort of an image, Gravatar is OK PyPI uses this for example (gravatar, not necessarily this lib) Usage: >>> g = Gravatar('myemailaddress@example.com') >>> g.get_image() 'https://www.gravatar.com/avatar/0bc83cb571cd1c50ba6f3e8a78ef1346' Brian #2: JSON to Pydantic Converter Suggested by Chun Ly, “this awesome JSON to @samuel_colvin's pydantic is so useful. It literally saved me days of work with a complex nested JSON schema.“ “JSON to Pydantic is a tool that lets you convert JSON objects into Pydantic models.” It’s a live site, where you can plop JSON on one the left, and Pydantic models show up on the right. There’s a couple options: Specify every field as Optional Alias camelCase fields as snake_case It’s

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1:distinctipy “distinctipy is a lightweight python package providing functions to generate colours that are visually distinct from one another.” Small, focused tool, but really cool. Say you need to plot a dynamic number of lines. Why not let distinctipy pick colors for you that will be distinct? Also can display the color swatches. Some example palettes here: https://github.com/alan-turing-institute/distinctipy/tree/main/examples from distinctipy import distinctipy # number of colours to generate N = 36 # generate N visually distinct colours colors = distinctipy.get_colors(N) # display the colours distinctipy.color_swatch(colors) Michael #2: Soda SQL Soda SQL is a free, open-source command-line tool. It utilizes user-defined input to prepare SQL queries that run tests on dataset in a data source to find invali

Watch the live stream: Watch on YouTube About the show Sponsored: RedHat: Compiler Podcast Special guest: Tonya Sims Michael #1: Pathy: a Path interface for local and cloud bucket storage via Spencer Pathy is a python package (with type annotations) for working with Cloud Bucket storage providers using a pathlib interface. It provides an easy-to-use API bundled with a CLI app for basic file operations between local files and remote buckets. It enables a smooth developer experience by letting developers work against the local file system during development and only switch over to live APIs for deployment. Also has optional local file caching. From Spenser The really cool function is "Pathy.fluid" which can take any type of local, GCS, or S3 path string and then just give you back a Path object that you can interact with agnostic of what platform it was. So this has worked amazingly for me in local testing since i can just change the file path from the "s3://bucket/path" that i use in prod to a local "

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1: pyscript Python in the browser, from Anaconda. repo here Announced at PyConUS “During a keynote speech at PyCon US 2022, Anaconda’s CEO Peter Wang unveiled quite a surprising project — PyScript. It is a JavaScript framework that allows users to create Python applications in the browser using a mix of Python and standard HTML. The project’s ultimate goal is to allow a much wider audience (for example, front-end developers) to benefit from the power of Python and its various libraries (statistical, ML/DL, etc.).” from a nice article on it, PyScript — unleash the power of Python in your browser PyScript is built on Pyodide, which is a port of CPython based on WebAssembly. Demos are cool. Note included in README: “This is an extremely experimental project, so expect things to break!” Michael #2: Memray from Bloomberg

Watch the live stream: Watch on YouTube About the show Sponsored: RedHat: Compiler Podcast Special guest: Anna Astori Michael #1: Take Your Github Repository To The Next Level

Watch the live stream: Watch on YouTube About the show Sponsored by Mergify! Special guest: Pat Decker Michael #0: New live stream / recording time: 12pm US PT on Tuesdays. Please subscribe to our YouTube channel to get notified and be part of the episodes. Brian #1: BTW, don’t make a public repo private How we lost 54k GitHub stars Jakub Roztočil HTTPie kinda sorta accidentally flipped their main repo to private for a sec. And dropped the star count from 54k to 0 oops They’re back up to 16k, as of today. But ouch. “HTTPie is a command-line HTTP client. Its goal is to make CLI interaction with web services as human-friendly as possible. HTTPie is designed for testing, debugging, and generally interacting with APIs & HTTP servers. The http & https commands allow for creating and sending arbitrary HTTP requests. They use simple and natural syntax and provide formatted and colorized output.” Actually, pretty cool tool to use for developing and testing APIs. Michael #2: The counter-intuitive

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Brian Skinn (Twitter | Github) Michael #1: OpenBB wants to be an open source challenger to Bloomberg Terminal OpenBB Terminal provides a modern Python-based integrated environment for investment research, that allows an average joe retail trader to leverage state-of-the-art Data Science and Machine Learning technologies. As a modern Python-based environment, OpenBBTerminal opens access to numerous Python data libraries in Data Science (Pandas, Numpy, Scipy, Jupyter) Machine Learning (Pytorch, Tensorflow, Sklearn, Flair) Data Acquisition (Beautiful Soup, and numerous third-party APIs) They have a discord community too BTW, seem to be a successful open source project: OpenBB Raises $8.5M in Seed Round Funding Following Open Source Project Gamestonk Terminal's Success Great graphics / gallery here. Way more affordable than the $1,900/mo/user for the Bloomberg Terminal Brian #2: Python f

Watch the live stream: Watch on YouTube About the show Sponsored by: Microsoft for Startups Founders Hub. Special guest: Vuyisile Ndlovu Brian #1: dunk - a prettier git diff Darren Burns Uses Rich “⚠️ This project is very early stages” - whatever, I like it. Recommendation is to use less as a pager for it git diff | dunk | less -R Michael #2: Is your Python code vulnerable to log injection? via Adam Parkin Let’s just appreciate log4jmemes.com for a moment Ok, now we can talk about Python We can freak our the logging with line injection "hello'.\nINFO:__main__:user 'alice' commented: 'I like pineapple pizza" Results in two lines for one statement INFO:__main__:user 'bob' commented: 'hello'. INFO:__main__:user 'alice' commented: 'I like pineapple pizza'. The safest solution is to simply not log untrusted text. If you need to store it for an audit trail, use a database. Alternatively, structured logging can prevent newline-based attacks. Padding a ton? One such case is abusing

Watch the live stream: Watch on YouTube About the show Sponsored by: Microsoft for Startups Founders Hub. Special guest: Thomas Gaigher, creator/maintainer pypyr taskrunner Michael #1: March Package Madness via Chris May Start with 16 packages They battle it out 2-on-2 in elimination rounds Voting is once a week So go vote! Brian #2: nbpreview “A terminal viewer for Jupyter notebooks. It’s like cat for ipynb files.” Some cool features pretty colors by default piping strips formatting, so you can pass it to grep or other post processing automatic paging syntax highlighting line numbers and wrapping work nicely markdown rendering images converted to block, character, or dots (braille) dataframe rendering clickable links Thomas #3: pyfakefs A fake file system! It intercepts all calls that involve the filesystem in Python - e.g open(), shutil, or pathlib.Path. This is completely transparent - your functional code does not know or need to know that under the hood it's been disconnected from

Watch the live stream: Watch on YouTube About the show Sponsored by FusionAuth: pythonbytes.fm/fusionauth Special guest: Ian Hellen Brian #1: gensim.parsing.preprocessing Problem I’m working on Turn a blog title into a possible url example: “Twisted and Testing Event Driven / Asynchronous Applications - Glyph” would like, perhaps: “twisted-testing-event-driven-asynchrounous-applications” Sub-problem: remove stop words ← this is the hard part I started with an article called Removing Stop Words from Strings in Python It covered how to do this with NLTK, Gensim, and SpaCy I was most successful with remove_stopwords() from Gensim from gensim.parsing.preprocessing import remove_stopwords It’s part of a gensim.parsing.preprocessing package I wonder what’s all in there? a treasure trove gensim.parsing.preprocessing.preprocess_string is one this function applies filters to a string, with the defaults almost being just what I want: strip_tags() strip_punctuation() strip_multiple_whitespaces() str

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Special guest: Emily Morehouse-Valcarcel Michael #1: Async and await with subprocesses by Fredrik Averpil People know I do all sorts of stuff with async Lots of cool async things are not necessarily built into Python, but our instead third-party packages E.g. files via aiofiles But asyncio has asyncio.create_subprocess_exec Fredrik’s article has some nice examples I started using this for mp3 uploads and behind the scenes processing for us Brian #2: Typesplainer Arian Mollik Wasi, @wasi_master Suggested by Will McGugan Now released a vscode extension for that! Available on vscode as typesplainer Emily #3: Ibis Project via Marlene Mhangami “Productivity-centric Python data analysis framework for SQL engines and Hadoop” focused on: Type safety Expressiveness Composability Familiarity Marlene wrote an excellent blog post as an introduction Works with tons of different backends, either dir

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Special guest: Anne Barela Brian #1: The Adam Test : 12 Questions for New Dependencies Found through a discussion with Ryan Cheley, who will be on an upcoming episode of Test & Code, talking about Managing Software Teams. The Joel Test dates back to 2000, and some of it is a bit dated. I should probably do a Test & Code episode or pythontest article on my opinions of this at some point. Nice shameless plugs, don’t you think? The Joel Test is 12 questions and is a “highly irresponsible, sloppy test to rate the quality of a software team.” “The Adam Test” is 12 questions “to decide whether a new package we’re considering depending on is well-maintained.” He’s calling it “The Well-Maintained Test”, but I like “The Adam Test” Here’s the test: Is it described as “production ready”? Is there sufficient documentation? Is there a changelog? Is someone responding to bug reports? Are there suf

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: Physics Breakthrough as AI Successfully Controls Plasma in Nuclear Fusion Experiment Interesting break through using AI Is Python at the center of it? With enough digging, the anwswer is yes, and we love it! Brian #2: PEP 680 -- tomllib: Support for Parsing TOML in the Standard Library Accepted for Python 3.11 This PEP proposes basing the standard library support for reading TOML on the third-party library tomli Michael #3: Thread local threading.local: A class that represents thread-local data. Thread-local data are data whose values are thread specific. Just create an instance of local (or a subclass) and store attributes on it You can even subclass it. Brian #4: What is a generator function? Trey Hunner Super handy, and way easier than you think, if you’ve never written your own. Really, it’s just a function that uses yield instead of return and supplies one element at a time

Watch the live stream: Watch on YouTube About the show Sponsor: Brought to you by FusionAuth - check them out at pythonbytes.fm/fusionauth Special guest: Calvin Hendryx-Parker Brian #1: Why your mock still doesn’t work Ned Batchelder Some back story: Why your mock doesn’t work a quick tour of Python name assignment The short version of Python Names and Values talk importing difference between from foo import bar and import foo w.r.t mocking punchline: “Mock it where it’s used” Now, Why your mock still doesn’t work talks about using @patch decorator (also applies to @patch.object decorator) and utilizing mock_thing1, mock_thing2 parameters to test you can change the return value or an attribute or whatever. normal mock stuff. But…. the punchline…. be careful about the order of patches. It needs to be @patch("foo.thing2") @patch("foo.thing1") def test_(mock_thing1, mock_thing2): ... Further reading: https://docs.python.org/3/library/unittest.mock.html#patch https://docs.pyt

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Steve Dower Michael #1: fastapi-events Asynchronous event dispatching/handling library for FastAPI and Starlette Features: straightforward API to emit events anywhere in your code events are handled after responses are returned (doesn't affect response time) support event piping to remote queues powerful built-in handlers to handle events locally and remotely coroutine functions (async def) are the first-class citizen write your handlers, never be limited to just what fastapi_events provides Brian #2: Ways I Use Testing as a Data Scientist Peter Baumgartner “In my work, writing tests serves three purposes: making sure things work, documenting my understanding, preventing future errors.” Test The results of some analysis process (using assert) Code that operates on data (using hypothesis) Aspects of the data (using pandera or Great Expectations

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Dean Langsam Brian #1: A Better Pygame Mainloop Glyph Doing some game programming is a great way to work on coding for early devs (and experienced devs). pygame is a popular package for writing games in Python But… the normal example of a main loop, which listens for events and dispatches actions based on events, has some problems: it’s got a while 1: that wastes power, too much busy waiting looks bad, due to “screen tearing” which is writing to a screen while your in the middle of drawing it This post discusses the problems, and walks through to an async main loop that creates a better gaming experience. Michael #2: awesome sqlalchemy A few notable ones SQLAlchemy-Continuum: Versioning and auditing extension for SQLAlchemy. SQLAlchemy-Utc: SQLAlchemy type to store aware datetime.datetime values. SQLAlchemy-Utils: Various utility functions, new data types and helpers for SQLAlchem

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Luciana Brian #1:rich-cli suggested by Lance Reinsmith rich on the command line. why? syntax highlighting rich example.py rich -m README.md use -m for markdown why Will? .md seems clear enough to me. comes with themes. ex: --theme monokai formats json, --json or -j and a bunch of other features I probably won’t use, but you might. alignment, maybe width, yeah, I’ll probably use -w a bunch more In my .zshrc: alias cat='rich --theme monokai' after pipx install rich-cli feel free to tell me that I shouldn’t used cat for looking at file contents. (although, why not?) I’m not, I’m using rich. :) Luciana #2: debugpy - a debugger for Python The debugger we use in the Python extension for VS Code Super heplful features that can save up a lot of time and a lot of folks don’t seem to know about: Conditional breakpoints Helpful when you want the code to break only on a specific co