Planet SciPy

My foreword to "Bayesian Analysis with Python, 2nd Edition" by Osvaldo Martin

When Osvaldo asked me to write the foreword to his new book I felt honored, excited, and a bit scared, so naturally I accepted. What follows is my best attempt to convey what makes probabilistic programming so exciting to me. Osvaldo did a great job with the book, it is …

Call for Answers: Survey About Task Assignment

Now Hiring: Events and Digital Marketing Coordinator

The post Now Hiring: Events and Digital Marketing Coordinator appeared first on NumFOCUS.

Revisiting authorship, and JOSS software publications

The question du jour: how should authorship on software papers be decided?

Using Bayesian Decision Making to Optimize Supply Chains

Gyroscopic 3D wireframe cube — Using a 3-axis gyro for live 3D perspective

This little project combines the previous accelerometer-gyroscope code with the 3D rotating OLED cube to produce a 3D cube which responds to gyro input, making it possible to "peek around" the cube with simulated perspective, or make it spin with a flick of the wrist.

Take a look at those …

Julia Co-Creators To Receive 2019 James H. Wilkinson Prize for Numerical Software

The post Julia Co-Creators To Receive 2019 James H. Wilkinson Prize for Numerical Software appeared first on NumFOCUS.

Mestrado em Ciência da Computação na UFPA 2019: Inteligência Computacional para Smart Grids; Metaheurísticas

LaKademy 2018

LaKademy 2018

GPU Dask Arrays, first steps

The following code creates and manipulates 2 TB of randomly generated data.

import dask.array as da

rs = da.random.RandomState()
x = rs.normal(10, 1, size=(500000, 500000), chunks=(10000, 10000))
(x + 1)[::2, ::2].sum().compute(scheduler='threads')

On a single CPU, this computation takes two hours.

On an eight-GPU single-node system this computation takes nineteen seconds.

Actually this computation isn’t that impressive. It’s a simple workload, for which most of the time is spent creating and destroying random data. The computation and communication patterns are simple, reflecting the simplicity commonly found in data processing workloads.

What is impressive is that we were able to create a distributed parallel GPU array quickly by composing these three existing libraries:

1. CuPy provides a partial implementation of Numpy on the GPU.

2. Dask Array provides chunked algorithms on top of Numpy-like libraries like Numpy and CuPy.

   This enables us to operate on more data than we could fit in memory by operating on that data in

3-axis Accelerometer-Gyro — Measuring acceleration and orientation with an MPU6050

Measuring acceleration and rotation has a lot of useful applications, from drone or rocket stablisation to making physically interactive handheld games.

An accelerometer measures proper acceleration, meaning the rate of change of velocity relative to it's own rest frame. This is in contrast to coordinate acceleration, which is relative to …

Manage project dependencies with conda environments

TL;DR: Create a conda environment for each project, capture exact versions when possible, automate activation and updating with a bash function.

I often work on several different projects involving software: Python libraries, papers, presentations, posters, this website, etc. Each project has different dependencies and there is a non-zero chance that these dependencies might be in conflict with each other. For example, I need Python 2.7 to work on a tesseroid modeling paper with a student, while my current work on

Anaconda Distribution 2018.12 Released

We are changing versioning in Anaconda Distribution from a major/minor version scheme to a year.month scheme. We made this change to differentiate between the open source Anaconda Distribution and Anaconda Enterprise, our managed data science platform. Conda, will continue to use a major/minor versioning scheme. The number of 32-bit x86 Linux packages downloaded are a …
Read more →

The post Anaconda Distribution 2018.12 Released appeared first on Anaconda.

From netlist to code: strategies to implement schematics modelling, the results

First Impressions of GPUs and PyData

I recently moved from Anaconda to NVIDIA within the RAPIDS team, which is building a PyData-friendly GPU-enabled data science stack. For my first week I explored some of the current challenges of working with GPUs in the PyData ecosystem. This post shares my first impressions and also outlines plans for near-term work.

First, lets start with the value proposition of GPUs, significant speed increases over traditional CPUs.

Like many PyData developers, I’m loosely aware that GPUs are sometimes fast, but don’t deal with them often enough to have strong feeling about them.

To get a more visceral feel for the performance differences, I logged into a GPU machine, opened up CuPy (a Numpy-like GPU library developed mostly by Chainer in Japan) and cuDF (a Pandas-like library in development at NVIDIA) and did a couple of small speed comparisons:

>>> import numpy, cupy

>>> x = numpy.random.random((10000, 10000))
>>> y = cupy.random.random((10000, 10000))

>>> %timeit bool((numpy.sin(x) ** 2 + numpy.cos(x) ** 2 == 1).all())
446 ms ± 53.1 ms per

Python Data Visualization 2018: Where Do We Go From Here?

This post is the third in a three-part series on the current state of Python data visualization and the trends that emerged from SciPy 2018. By James A. Bednar As we saw in Part I and Part II of this series, having so many separate Python visualization libraries to choose from often is confusing to …
Read more →

The post Python Data Visualization 2018: Where Do We Go From Here? appeared first on Anaconda.

Double Your Donation: NumFOCUS Receives $5,000 Matching Gift Challenge

The post Double Your Donation: NumFOCUS Receives $5,000 Matching Gift Challenge appeared first on NumFOCUS.

Fun review: the Lego Rough Terrain Crane

A quick read of _The genomic and proteomic landscape of the rumen microbiome_

Using short and long reads to assemble genomes from metagenomes!

Intake for Cataloging Spark

By: Martin Durant Intake is an open source project for providing easy pythonic access to a wide variety of data formats, and a simple cataloging system for these data sources. Intake is a new project, and all are encouraged to try and comment on it. pySpark is the python interface to Apache Spark, a fast …
Read more →

The post Intake for Cataloging Spark appeared first on Anaconda.

Highlights from the 2018 NYC DISC Sprint

The post Highlights from the 2018 NYC DISC Sprint appeared first on NumFOCUS.

Using Pip in a Conda Environment

Unfortunately, issues can arise when conda and pip are used together to create an environment, especially when the tools are used back-to-back multiple times, establishing a state that can be hard to reproduce. Most of these issues stem from that fact that conda, like other package managers, has limited abilities to control packages it did …
Read more →

The post Using Pip in a Conda Environment appeared first on Anaconda.

Book review: Introduction to Electrical Circuit Analysis

Python Data Visualization 2018: Moving Toward Convergence

By James A. Bednar This post is the second in a three-part series on the current state of Python data visualization and the trends that emerged from SciPy 2018. In my previous post, I provided an overview of the myriad Python data visualization tools currently available, how they relate to each other, and their many …
Read more →

The post Python Data Visualization 2018: Moving Toward Convergence appeared first on Anaconda.

Understanding Conda and Pip

Conda and pip are often considered as being nearly identical. Although some of the functionality of these two tools overlap, they were designed and should be used for different purposes. Pip is the Python Packaging Authority’s recommended tool for installing packages from the Python Package Index, PyPI. Pip installs Python software packaged as wheels or …
Read more →

The post Understanding Conda and Pip appeared first on Anaconda.

Support Python 2 with Cython

Many popular Python packages are dropping support for Python 2 next month. This will be painful for several large institutions. Cython can provide a temporary fix by letting us compile a Python 3 codebase into something usable by Python 2 in many cases.

It’s not clear if we should do this, but it’s an interesting and little known feature of Cython.

Many major numeric Python packages are dropping support for Python 2 at the end of this year. This includes packages like Numpy, Pandas, and Scikit-Learn. Jupyter already dropped Python 2 earlier this year.

For most developers in the ecosystem this isn’t a problem. Most of our packages are Python-3 compatible and we’ve learned how to switch libraries. However, for larger companies or government organizations it’s often far harder to switch. The PyCon 2017 keynote by Lisa Guo and Hui Ding from Instagram gives a good look into why this can be challenging for large production codebases and also gives a good

This #GivingTuesday, We’re Saying “Thank You” To Our Supporters

The post This #GivingTuesday, We’re Saying “Thank You” To Our Supporters appeared first on NumFOCUS.

Monitoring embedded space quality for classification

Anatomy of an OSS Institutional Visit

I recently visited the UK Meteorology Office, a moderately large organization that serves the weather and climate forecasting needs of the UK (and several other nations). I was there with other open source colleagues including Joe Hamman and Ryan May from open source projects like Dask, Xarray, JupyterHub, MetPy, Cartopy, and the broader Pangeo community.

This visit was like many other visits I’ve had over the years that are centered around showing open source tooling to large institutions, so I thought I’d write about it in hopes that it helps other people in this situation in the future.

My goals for these visits are the following:

1. Teach the institution about software projects and approaches that may help them to have a more positive impact on the world
2. Engage them in those software projects and hopefully spread around the maintenance and feature development burden a bit

We were invited by early adopters within the institution, both within the UK Met Office’s Informatics Lab

Deriving Business Value from Data Science Deployments

By Gus Cavanaugh One of the biggest challenges facing organizations trying to derive value from data science and machine learning is deployment. In this post, we’ll take a look at three common approaches to deploying data science projects, and how Anaconda Enterprise simplifies deployment and allows data scientists to focus on building better models that …
Read more →

The post Deriving Business Value from Data Science Deployments appeared first on Anaconda.

From netlist to code: strategies to implement schematics modelling

NumFOCUS Funds Open Source Projects through Small Development Grants – Fall 2018

The post NumFOCUS Funds Open Source Projects through Small Development Grants – Fall 2018 appeared first on NumFOCUS.

The story behind the spacegraphcats project and paper

The TRUE story!

SunPy Maps: Digitizing Images of the Sun from the 1970s

The post SunPy Maps: Digitizing Images of the Sun from the 1970s appeared first on NumFOCUS.

Python Data Visualization 2018: Why So Many Libraries?

This post is the first in a three-part series on the state of Python data visualization tools and the trends that emerged from SciPy 2018. By James A. Bednar At a special session of SciPy 2018 in Austin, representatives of a wide range of open-source Python visualization tools shared their visions for the future of …
Read more →

The post Python Data Visualization 2018: Why So Many Libraries? appeared first on Anaconda.

Notes on the Frank-Wolfe Algorithm, Part II: A Primal-dual Analysis

This blog post extends the convergence theory from the first part of my notes on the Frank-Wolfe (FW) algorithm with convergence guarantees on the primal-dual gap which generalize and strengthen the convergence guarantees obtained in the first part.

Fun review: the Lego Bugatti Chiron

Creating a welcoming teaching/learning environment in workshops

It takes constant work to make a welcoming teaching/learning environment!

Choose Your Anaconda IDE Adventure: Jupyter, JupyterLab, or Apache Zeppelin

As humans we are faced with multiple choices every day. Every person is different: some people prefer Firefox while others like Chrome; some people prefer Python while others like R. Here at Anaconda, we abstain from engaging in language or IDE wars, and firmly believe our users shouldn’t have to compromise their preferences. That’s why …
Read more →

The post Choose Your Anaconda IDE Adventure: Jupyter, JupyterLab, or Apache Zeppelin appeared first on Anaconda.

Repeatability in Practice (2018 version)

How we do repeatability in the DIB Lab

Who You Gonna Call? Halloween Tips & Treats to Protect You from Ghosts, Gremlins…and Software Vulnerabilities

By Michael Sarahan Happy Halloween, readers. At Anaconda, we’re not too scared about things that go bump in the night. We’ve examined the data and concluded that it’s just the cleaning staff upstairs. We are, however, kept awake by the ever-present concern of the security and experience of our users! We’d like to take this …
Read more →

The post Who You Gonna Call? Halloween Tips & Treats to Protect You from Ghosts, Gremlins…and Software Vulnerabilities appeared first on Anaconda.

Linking to emails in org-mode (using neomutt)

Where we store links to emails in org-mode, and open them using neomutt.

Ode ao ódio

Eleições 2018: Minha carta para a família

A arquitetura de compartilhamentos do Telegram para mitigar as fake news no WhatsApp

Audio ToolKit: Moving to C++17

NumFOCUS Announces NVIDIA as Gold Sponsor

The post NumFOCUS Announces NVIDIA as Gold Sponsor appeared first on NumFOCUS.

So you want to contribute to open source

Welcome new open source contributor!

I appreciated receiving the e-mail where you said you were excited about getting into open source and were particularly interested in working on a project that I maintain. This post has a few thoughts on the topic.

First, please forgive me for sending you to this post rather than responding with a personal e-mail. Your situation is common today, so I thought I’d write up thoughts in a public place, rather than respond personally.

This post has two parts:

1. Some pragmatic steps on how to get started
2. A personal recommendation to think twice about where you focus your time

Most open source software (OSS) projects have a “Good first issue” label on their Github issue tracker. Here is a screenshot of how to find the “good first issue” label on the Pandas project:

(note that this may be named something else like “Easy to fix”)

This contains a list of issues that are important, but also

What Happened at the NumFOCUS Summit 2018?

The post What Happened at the NumFOCUS Summit 2018? appeared first on NumFOCUS.

Updates to the NumFOCUS Code of Conduct

The post Updates to the NumFOCUS Code of Conduct appeared first on NumFOCUS.

Dictionary Views & Set Operations — Working with dictionary view objects

The keys, values and items from a dictionary can be accessed using the .keys(), .values() and .items() methods. These methods return view objects which provide a view on the source dictionary.

The view objects dict_keys and dict_items support set-like operations (the latter only when all values are hashable) which …

Ciro em frente!

Inaugural NumFOCUS Awards and New Contributor Recognition

The post Inaugural NumFOCUS Awards and New Contributor Recognition appeared first on NumFOCUS.

Successful Sustainer Weeks Fundraiser for Open Source Scientific Computing Software

The post Successful Sustainer Weeks Fundraiser for Open Source Scientific Computing Software appeared first on NumFOCUS.

Announcement: Audio TK 3.0.0

3D wireframe cube with MicroPython — Basic 3D model rotation and projection

An ESP2866 is never going to compete with an actual graphics card. But it has more than enough oomph to explore the fundamentals of 3D graphics. In this short tutorial we'll go through the basics of creating a 3D scene and displaying it on an OLED screen using MicroPython.

This …

Akademy 2018

QtConsole 4.4 Released!

We're excited to announce a significant update to QtConsole—the package that powers Spyder's IPython Console interface—which the Spyder team maintains in collaboration with Project Jupyter. Two of the biggest changes—user-selectable syntax highlighting themes, and enhanced external editor/IDE integration—are already built right into Spyder, so they'll likely be of more interest if you use QtConsole standalone or with another editor/IDE. However, most of the other changes should prove quite useful within Spyder as well, and many were in fact suggested and even implemented by users of our IDE. Particular highlights include a block indent/unindent feature, Select-All (Ctrl-Shift-A) being made cell-specific, Ctrl-Backspace and Ctrl-Delete behaving more intelligently across whitespace and line boundaries, Ctrl-D allowing you to easily exit ipdb, input() and the like, and numerous smaller enhancements and bug fixes. If you'd like to learn more about what's new, please check out our article over on the Jupyter blog, where we go over the major changes in more detail, with plenty

Book review: Continuous Delivery With Docker And Jenkins

Compiling C++ code in memory with clang

Dask Development Log

This work is supported by Anaconda Inc

To increase transparency I’m trying to blog more often about the current work going on around Dask and related projects. Nothing here is ready for production. This blogpost is written in haste, so refined polish should not be expected.

Since the last update in the 0.19.0 release blogpost two weeks ago we’ve seen activity in the following areas:

1. Update Dask examples to use JupyterLab on Binder
2. Render Dask examples into static HTML pages for easier viewing
3. Consolidate and unify disparate documentation
4. Retire the hdfs3 library in favor of the solution in Apache Arrow.
5. Continue work on hyper-parameter selection for incrementally trained models
6. Publish two small bugfix releases
7. Blogpost from the Pangeo community about combining Binder with Dask
8. Skein/Yarn Update

The new dask-labextension embeds Dask’s dashboard plots into a JupyterLab session so that you can get easy access to information

A foundation for scikit-learn at Inria

We have just announced that a foundation will be supporting scikit-learn at Inria [1]: scikit-learn.fondation-inria.fr

This is an exciting turn for us, because it enables us to receive private funding. As a result, we will be able to have secure employment for some existing core …

Introducing Verde

Verde is a Python library for processing spatial data (bathymetry, geophysics surveys, etc) and interpolating it on regular grids (i.e., gridding).

It implements Green's functions based interpolation methods and other data processing routines. The type of gridding implemented in Verde is essentially fitting various linear models to spatial data and using them to predict new data on regular grids, which is what a lot of machine learning is all about. So Verde's gridder API is inspired on scikit-learn, the state-of-the-art for machine learning in Python. The Green's functions that make up the Jacobian matrix (aka sensitivity or feature matrix) of the linear models generally come from elastic deformation theory. For example, the bi-harmonic spline (Sandwell, 1987) implemented in verde.Spline comes from the deformation of a thin elastic plate.

I submitted a

The Waiting Time Paradox, or, Why Is My Bus Always Late?

Image Source: Wikipedia License CC-BY-SA 3.0

If you, like me, frequently commute via public transit, you may be familiar with the following situation:

You arrive at the bus stop, ready to catch your bus: a line that advertises arrivals every 10 minutes. You glance at your watch and note the time... and when the bus finally comes 11 minutes later, you wonder why you always seem to be so unlucky.

Naïvely, you might expect that if buses are coming every 10 minutes and you arrive at a random time, your average wait would be something like 5 minutes. In reality, though, buses do not arrive exactly on schedule, and so you might wait longer. It turns out that under some reasonable assumptions, you can reach a startling conclusion:

When waiting for a bus that comes on average every 10 minutes, your average waiting time will be 10 minutes.

This is what is sometimes known as the waiting time paradox.

I've encountered this idea before, and always wondered

Akademy 2018

Three Operator Splitting

Dask Release 0.19.0

This work is supported by Anaconda Inc.

I’m pleased to announce the release of Dask version 0.19.0. This is a major release with bug fixes and new features. The last release was 0.18.2 on July 23rd. This blogpost outlines notable changes since the last release blogpost for 0.18.0 on June 14th.

You can conda install Dask:

conda install dask

or pip install from PyPI:

pip install dask[complete] --upgrade

Full changelogs are available here:

• dask/dask
• dask/distributed

A ton of work has happened over the past two months, but most of the changes are small and diffuse. Stability, feature parity with upstream libraries (like Numpy and Pandas), and performance have all significantly improved, but in ways that are difficult to condense into blogpost form.

That being said, here are a few of the more exciting changes in the new release.

We’ve dropped official support for Python 3.4 and added official support for Python 3.7.

Over the past few months Jim Crist has bulit a suite of

Summer school announcement: 2nd Advanced Scientific Programming in Python (ASPP) Asia Pacific!

Abstract for SIAM: Supporting and Sustaining Open Source Software Development: the Commons Perspective

How do we support and sustain open source software development?

High level performance of Pandas, Dask, Spark, and Arrow

This work is supported by Anaconda Inc

How does Dask dataframe performance compare to Pandas? Also, what about Spark dataframes and what about Arrow? How do they compare?

I get this question every few weeks. This post is to avoid repetition.

1. This answer is likely to change over time. I’m writing this in August 2018
2. This question and answer are very high level. More technical answers are possible, but not contained here.

If you’re coming from Python and have smallish datasets then Pandas is the right choice. It’s usable, widely understood, efficient, and well maintained.

The performance benefit (or drawback) of using a parallel dataframe like Dask dataframes or Spark dataframes over Pandas will differ based on the kinds of computations you do:

1. If you’re doing small computations then Pandas is always the right choice. The administrative costs of parallelizing will outweigh any benefit. You should not parallelize if your computations are taking less

Displaying images on OLED screens — Using 1-bpp images in MicroPython

We've previously covered the basics of driving OLED I2C displays from MicroPython, including simple graphics commands and text. Here we look at displaying monochrome 1 bit-per-pixel images and animations using MicroPython on a Wemos D1.

Processing the images and correct choice of image-formats is important to get the most detail …

Dictionaries — An rather long guide to Python's key:value store

Dictionaries are key-value stores, meaning they store, and allow retrieval of data (or values) through a unique key. This is analogous with a real dictionary where you look up definitions (data) using a given key — the word. Unlike a language dictionary however, keys in Python dictionaries are not alphabetically sorted …

Driving I2C OLED displays with MicroPython — I2C monochrome displays with SSD1306

These mini monochrome OLED screens make great displays for projects — perfect for data readout, simple UIs or monochrome games.

Raindar — Desktop daily weather, forecast app in PyQt

The Raindar UI was created using Qt Designer, and saved as .ui file, which is available for download. This was converted to an importable Python file using pyuic5.

Before running the application you need to obtain a API key from OpenWeatherMap.org. This key is unique to you …

Public Institutions and Open Source Software

As general purpose open source software displaces domain-specific all-in-one solutions, many institutions are re-assessing how they build and maintain software to support their users. This is true across for-profit enterprises, government agencies, universities, and home-grown communities.

While this shift brings opportunities for growth and efficiency, it also raises questions and challenges about how these institutions should best serve their communities as they grow increasingly dependent on software developed and controlled outside of their organization.

• How do they ensure that this software will persist for many years?
• How do they influence this software to better serve the needs of their users?
• How do they transition users from previous all-in-one solutions to a new open source platform?
• How do they continue to employ their existing employees who have historically maintained software in this field?
• If they have a mandate to support this field, what is the best role for them to play, and how can they justify their efforts to the groups that control their budget?

This blogpost

Can bits be the basis for a digital commons? (No.)

Bits cannot be the basis for a digital commons, because they are not rivalrous.

Spyder 3.3.0 and 3.3.1 released!

We're pleased to release the next significant update in the stable Spyder 3 line, 3.3.0, along with its follow-on bugfix point release, 3.3.1, which is now live on PyPI and conda. As always, you can update with conda update spyder in the Anaconda Prompt/Terminal/command line (on Windows/macOS/Linux, respectively) if on Anaconda (recommended), or pip update spyder otherwise. If you run into any trouble, please carefully read our new installation documentation and consult our Troubleshooting Guide, which contains straightforward solutions to the vast majority of install-related issues users have reported.

As a new minor version (3.3), it makes several substantial changes to Spyder's underpinnings that deserve some explanation, particularly the newly modular and portable console system that's now separated into its own spyder-kernels package, opening up several new options for users running Spyder in different environments. There's also a brand-new error reporting process, new options in the IPython console, usability and performance improvements for the Variable Explorer, multiple new and changed dependency requirements

Hierarchical Bayesian Neural Networks with Informative Priors

Spyder featured on Episode 1 of Open Source Directions web show

Quansight, the company recently founded by NumPy, SciPy and Anaconda creator Travis Oliphant to help connect companies with open source communities built around data science and machine learning, just released Episode 1 of its live webcast series, and it was all about Spyder! Spyder maintainer Carlos Córdoba, recently hired by Quansight and funded part-time to work on Spyder development as we announced a few weeks ago, was the featured guest on the show.

Carlos first shared his perspective on some of the key moments in Spyder's nearly 10-year development history, from its original creation by Pierre Raybaut and Carlos' initial involvement in the project to its more recent challenges and successes. He also demonstrated basic usage of Spyder, as well as some of its standout features, in a live on-screen demo. Carlos then went on to outline the current roadmap for Spyder 4 in the near future, and explained some of the key new features planned for it. Finally, he took

NeuroML2/LEMS is moving into Neural Mass Models and whole brain networks

"Labor" and "Engaged effort"

Are "effort" and "labor" the same?

Sprint on scikit-learn, in Paris and Austin

Two weeks ago, we held a scikit-learn sprint in Austin and Paris. Here is a brief report, on progresses and challenges.

Websites for Earth Scientists on the academic job hunt

This is a list of the websites I use to search for academic jobs in the Earth Sciences (geophysics, geology, oceanography, meteorology, etc). They've been very useful to me (I found my current position through the CIG mailing list) and I hope that this post can help others who are looking to take the next step in their academic careers.

These sites list everything from Masters and PhD scholarships to postdoc positions and tenure-track professorships. Note that they are biased toward the US, Canada, Oceania, and Europe.

Sign up for these and get email updates when new opportunities are posted (most are updated daily):

• ES_JOBS_NET: I get around 10 emails from this list a day. Lately, I'm seeing a lot of

State of the Spyder, Part 2: Looking up

After sharing some major milestones, development progress, and other tidbits from the past six months in Part 1 of this series (check that one out first if you haven't already), we now have some amazing news to share with you all here in Part 2, along with other status updates. That's not all, though—Part 3 will look ahead toward Spyder 4 and beyond, unveiling and explaining our full roadmap and going over the future possibilities even further afield.

First up, we're thrilled to announce a major part of what's making that plan possible (along with your support, of course!). This May, Spyder was awarded a $3000 development grant from NumFOCUS, an organization promoting better science through open code, to help with finishing Spyder 4! NumFOCUS is a nonprofit dedicated to supporting key scientific computing projects; promoting sustainability in the open source ecosystem; educating the next generation of scientists, engineers, developers and data analysts through their flagship

Introducing Pooch

A friend to fetch your sample data files.

Pooch is a Python package that manages downloading data files over HTTP and storing them in a local directory. It is meant to be used by other Python libraries that ship sample data files for use in documentation, workshops, demos, etc.

For example, your package could define a datasets.py module that has functions to load sample data (like scikit-learn does). If you want the data to live on the web (like in the Github repo) instead of shipping it with your package, Pooch can keep track of it and download it to the user's computer only when it's needed.

This is what a datasets.py module would look like using Pooch:

"""
Module mypackage/datasets.py
"""
import pooch

# Get the version string from your project. You have one of these, right?
from

The road to scikit-image 1.0

The Open Source Anti-Sisyphean League

We need an Open Source Anti-Sisyphean League!

Does batting order matter in Major League Baseball? A simulation approach

A framework for thinking about Open Source Sustainability?

Can we apply Common Pool Resource work to open online projects?

What do scientists know about open source?

Get in it

Two weeks ago, Project Jupyter had our only planned team meeting for 2018. There was too much stuff going on for me to write a poem during the event as I had in previous years (2016, and 2017), so I ended up reading one of the pieces I wrote during my evening introvert breaks in Cleveland at PyCon a few weeks earlier.

Once again, Fernando and Matthias had their gadgets ready to record (thank you both!). The video below was taken by Fernando.

Time suspended
Gellatinous reality - the haze
submerged in murky drops summed
in swamp pond of life

believe and strive, expand the mind
A state sublime, when in your prime you came to
me and we were free to flow and fling our
cares, our dreams, our in-betweens, our
rêves perdues, our residue -- the lime of light
the black of sight -- all these converge and
merge the forks of friction filled with fright
and more -- the float of logs that plunges deep
beyond the fray, beyond the

7Pez — Desktop unzip application with custom window decoration

This is a functionally terrible unzip application, saved only by the fact that you get to look at a cat while using it.

The original idea reflected in the name 7Pez was actually worse — to rig it up so you had to push on the head to unzip each file …

Failamp — Multimedia playlist & player in Python, using PyQt

Failamp is a simple audio & video mediaplayer implemented in Python, using the built-in Qt playlist and media handling features. It is modelled, very loosely on the original Winamp, although nowhere near as complete (hence the fail).

The main window UI was built using Qt Designer. The screenshot …

Creating a window with PyQt5 — The first step in creating your GUI application

The first step in creating desktop applications with PyQt is getting a window to show up on your desktop. Thankfully, with PyQt that is pretty simple.

Below are a few short examples to creating PyQt apps and getting a window on the screen. If this works you know you have …

QtWebEngineWidgets, the new browser API in PyQt 5.6 — Simplified page model and asynchronous methods

With the release of Qt 5.5 the Qt WebKit API was deprecated and replaced with the new QtWebEngine API, based on Chromium. The WebKit API was subsequently removed from Qt entirely with the release of Qt 5.6 in mid-2016.

The change to use Chromium for web widgets within …

Brown Note — Desktop notes app using SQLAlchemy & PyQt

Relieve your creative blockages with these interactive desktop reminders.

Brown Note is a desktop notes application written in Python, using PyQt. The notes are implemented as decoration-less windows, which can be dragged around the desktop and edited. Details in the notes, and their position on the desktop, is stored in …

Quick followups: NGLess benchmark & Notebooks as papers

Lucky Cat Spinning-arm Display — Python-powered Maneki-neko persistence of vision scroller

This build started as something simple: a lucky cat which would turn on and off automatically in response to some event. Since lucky cats are associated with good fortune the idea was to make one do this every time I got paid. This was working pretty well but unfortunately, after …

NSAViewer — Webcam viewer & photo booth in Python, using PyQt

This app isn't actually a direct line from your webcam to the NSA, it's a demo of using the webcam/camera support in Qt. The name is a nod to the paranoia (or is it...) of being watched through your webcam by government spooks.

I did consider making the app …