Planet SciPy
What to do when you have small dataset
I'm trying to train a classifier with a neural network, but I've got too small datasets. Each class has about ~1k examples. What is the best approach?...Why network overfits too early?
I want to train a neural network model, which basically does binary classification. I can't understand why my network overfits too early. I thought my network is too big and it memorizes the dataset, but when I make it smaller, it does not learn at all. How avoid this situation? Dropout didn't work, augmentation techniques helped a bit, obviously, regularizations didn't change anything. Can you guys explain the reasons, and how I can avoid it?...Multiple cuda versions installed in machine
I'm using TensorFlow and I have some old projects which are written with TensorFlow 1.4 and older. Some of them don't work with a new version of Cuda . Can I have multiple Cuda with different versions at the same time?...Cuda Version
How to know the version of Cuda installed on your pc? I'm using Keras with TensorFlow back-end, but I need to detect the version of Cuda in my code. It does not matter the solution is with Keras or TensorFlow....Dynamic learning rate in training
I'm using Keras 2.1.* and want to change the learning rate during training. I know about the scheduled callback, but I don't use the fit function and I don't have callbacks. I use train_on_batch. Is it possible in Keras?...Explainability and Auditability in ML: Definitions, Techniques, and Tools
Imagine that you have to present your newly built facial recognition feature to the technical heads of a SaaS product. The presentation goes relatively well until the CTO asks you “so what exactly goes on inside?” and all you can say is “nobody knows, it’s a black box”. Pretty soon, other stakeholders would start to […]
The post Explainability and Auditability in ML: Definitions, Techniques, and Tools appeared first on neptune.ai.
How To Convert Python List To Pandas DataFrame
Pandas dataframe is a very useful data structure.
In this notebook, I will show with examples how to convert Python List to Pandas Dataframe.
import pandas as pd
Let us create a dummy list of stock symbols.
stocks = ['AMC', 'GME', 'BB', 'CLOV', 'PLTR']
Creating Dataframe from list can be achieved using pandas.DataFrame.
df = pd.DataFrame(stocks,columns=['ticker'])
Let us look at our dataframe now.
df.head()
| ticker | |
|---|---|
| 0 | AMC |
| 1 | GME |
Moving sourmash towards more community engagement - a funding application
CZI EOSS4 application for sourmash support
Visualization with Seaborn
This article will enable you to use the seaborn python package to visualize your structured data with seaborn barchart, scatter plot, seaborn histogram, line, and seaborn distplot.Packaging ML Models: Web Frameworks and MLOps
In this article, we’ll see the what, why and how, of the top packaging tools – web-based frameworks and MLOps – for Data Science and ML projects. Data scientists and machine learning engineers need specific tools for building, deploying and monitoring these projects end-to-end. We’ll go through several tools in detail, along with their key […]
The post Packaging ML Models: Web Frameworks and MLOps appeared first on neptune.ai.
Searching all public metagenomes with sourmash
Searching all the things!
Create applications with QtQuick — Build modern applications with declarative QML
In previous tutorials we've used the Qt Widgets API for building our applications. This has been the standard method for building applications since Qt was first developed. However, Qt provides another API for building user interfaces: Qt Quick. This is a modern mobile-focused API for app development, with which you …
How Businesses Can Support Open-Source Communities
Businesses can support the broader open-source community, rather than just individual projects, by investing in things like conferences and access to hardware for different developer groups. This helps strengthen the community as a whole and supports the overall innovation commons. For example, one way we’ve chosen to do this at Anaconda is through the Anaconda Dividend Program. It works by donating 1% of our Commercial Edition revenue to the NumFOCUS organization to support various open-source projects and programs. Although we continue to also contribute to projects on a more ad-hoc basis, we felt it was important to establish a way to invest in open source that would support the sustainable, long-term growth of the innovation commons.A Comprehensive Guide On How to Monitor Your Models in Production
*Record scratch* *Freeze frame* Yup, that’s me being plowed to the ground because the business just lost more than $500,000 with our fraud detection system by wrongly flagging fraudulent transactions as legitimate, and my boss’s career is probably over. And that guy in the chair? That’s our DevOps Engineer. You’re probably wondering how we got […]
The post A Comprehensive Guide On How to Monitor Your Models in Production appeared first on neptune.ai.
MLOps Problems and Best Practices
You, me, all of us generate tons of data for every minute spent online. For business and science alike, this is an opportunity that can’t be overlooked. The hype for AI and Machine Learning keeps ramping up, as more organizations adopt these technologies. The amount of user-generated data has grown exponentially. Traditional on-premise servers transformed […]
The post MLOps Problems and Best Practices appeared first on neptune.ai.
Best MLOps Tools for Your Computer Vision Project Pipeline
The lifecycle of an app or software system (also known as SDLC) has several main stages: Planning, Development, Testing, Deployment, Then again, back to new releases with features, updates, and/or fixes as needed. To carry out these processes, software development relies on DevOps to streamline development while continuously delivering new releases and maintaining quality. The […]
The post Best MLOps Tools for Your Computer Vision Project Pipeline appeared first on neptune.ai.
When MLOps Is an Organizational and Communication Problem – Not a Tech Problem
In this article, you will get a compact overview of MLOps, its stages. You will also get a walkthrough of instances when MLOps is an organizational and communication problem and when it is a tech problem and how to resolve these challenges. What is MLOps? MLOps is closely inspired by the concept of DevOps where […]
The post When MLOps Is an Organizational and Communication Problem – Not a Tech Problem appeared first on neptune.ai.
The Ongoing Journey of AAPI Representation in Tech
This year’s Asian American and Pacific Islander Heritage Month comes at a pivotal time for the community. On the one hand, there has never been more public dialogue about diversity and inclusion, including those issues affecting the AAPI community. Yet, we have also seen a troubling rise in violence and prejudice against Asian Americans in recent months. While there are no simple solutions to these challenges, AAPI Heritage month provides an opportunity to uplift AAPI voices and encourage Asian Americans to share their stories. As a first-generation Chinese American and co-founder of Anaconda, I believe I am responsible for sharing my perspective, particularly about the Asian American experience in the technology field.How These 8 Companies Implement MLOps – In-Depth Guide
You’ve probably seen the (not so recent, but still true) news: Yup! Unfortunately, there are a lot of cases where companies try to operationalize their ML projects in a way that makes sense to their business but never really reach a successful implementation. The last mile for AI project success is the deployment and management […]
The post How These 8 Companies Implement MLOps – In-Depth Guide appeared first on neptune.ai.
Segmenting and Colorizing Images in IOS App Using Deoldify and Django API
Image segmentation falls under the field of imaging involving deep object detection and recognition. If we segregate an image into multiple regions by separating pixel-wise, each object in the scene allows us to train sophisticated deep learning models for tasks that require high standards of image analysis and context interpretation. Models trained this way can […]
The post Segmenting and Colorizing Images in IOS App Using Deoldify and Django API appeared first on neptune.ai.
Top Machine Learning Startups to Watch in 2021
Machine learning has come a long way. After decades of research, machine learning went mainstream in 2012 when an AI solution won the ImageNet challenge by a whopping margin of 10.8%, or 41% better than the runner-up score! From very limited usage in the business world before 2012, machine learning dependency has gone up exponentially […]
The post Top Machine Learning Startups to Watch in 2021 appeared first on neptune.ai.
Putting out the fire: Where do we start with accessibility in JupyterLab?
I want to be honest with you, I started asking accessibility questions in JupyterLab spaces while filled with anxiety. Anxiety that I was shouting into the void and no one else would work on accessibility with me. Anxiety that I didn’t have the skills or energy or knowledge to back up what I wanted to do. Anxiety that I was going to do it wrong and make JupyterLab even more inaccessible. Sometimes I still feel that way.
Read more… (6 min remaining to read)
Understanding of Probability Distribution and Normal Distribution
Introduction of probability distribution and its types. Here you can find the intuition about the normal or gaussian distribution, standard normal distribution with the normal curve and normal distribution formula.Model Registry Makes MLOps Work – Here’s Why
Model Registry is a part of the machine learning lifecycle or MLOps. It is a service that manages multiple model artifacts, tracks, and governs models at different stages of the ML lifecycle. Model registry is a collaborative hub where teams can work together at different stages of the machine learning lifecycle, starting from the experimentation […]
The post Model Registry Makes MLOps Work – Here’s Why appeared first on neptune.ai.
Is your software ready for the Journal of Open Source Software?
For the unaware reader, the Journal of Open Source Software (JOSS) is an open-access scientific journal founded in 2016 and aimed at publishing scientific software. A JOSS article in itself is short and its publication contributes to recognize the work on the software. I share here my point of view on what makes some software tools more ready to be published in JOSS. I do not comment on the size or the relevance for research which are both documented on JOSS' website.
Introduction of Fast Fourier Transformation (FFT)
This article comprises of introduction to the Fourier series, Fourier analysis, Fourier transformation, why do we use it, an explanation of the FFT algorithm, and its implementation.Anaconda Brings Data Science to Linux on IBM Z and LinuxONE
Today, along with IBM, we are excited to announce the general availability of Anaconda for Linux on Z and LinuxONE. Anaconda on Linux on Z and LinuxONE is our latest step toward expanding the availability of key open-source data science tools across platforms and improving the experience for practitioners everywhere.Using Autoencoder to generate digits with Keras
This article contains a real-time implementation of an autoencoder which we will train and evaluate using very known public benchmark dataset called MNIST data.Understanding of Support Vector Machine (SVM)
Explanation of the support vector machine algorithm, the types, how it works, and its implementation using the python programming language with the sklearn machine learning packageBasic Counting in Python
I love fancy machine learning algorithms as much as anyone. But sometimes, you just need to count things. And Python’s built-in data structures make this really easy. Let’s say we have a list of strings: With a list like this, you might care about a few different counts. What’s the ... Read More
The post Basic Counting in Python appeared first on Sparrow Computing.
Confidence Interval Understanding
Explanation of confidence intervals and the how-to calculate it for different scenarios, and also the equation that makes the confidence interval and the parameters involved with itDecision Trees
Intuition and implementation of the first tree-based algorithm in machine learningDimensionality Reduction, PCA Intro
We will be covering a dimensionality reduction algorithm called PCA (Principal Components Analysis) and will show how it helps to understand the data you have.Announcing Anaconda Support on AWS Graviton2
Today, Anaconda, the world’s most popular Python distribution platform for data science and machine learning, in collaboration with AWS, is happy to announce the general availability of Anaconda for Linux on the aarch64 (arm64) platform optimized for Amazon Web Services’ Graviton2 processors.Anaconda Individual Edition 2021.05
There were also quite a few other bug fixes and improvements - to see the full list please visit the Anaconda Navigator 2.0.3 release notes here.PySide tutorial now available — Complete course, updated for PySide2 & PySide6
Hello! With the release of Qt6 versions of PyQt and PySide the course was getting a little crowded. So, today I've split the PySide tutorials into their own standalone PySide course.
The tutorials have all been updated for PySide2 & PySide6, with some additional improvements based on the latest editions of …
Understanding Autoencoders - An Unsupervised Learning approach
This article covers the concept of Autoencoders. Concepts like What are Autoencoders, Architecture of an Autoencoder, and intuition behind the training of Autoencoders.How to Use the PyTorch Sigmoid Operation
The PyTorch sigmoid function is an element-wise operation that squishes any real number into a range between 0 and 1. This is a very common activation function to use as the last layer of binary classifiers (including logistic regression) because it lets you treat model predictions like probabilities that their ... Read More
The post How to Use the PyTorch Sigmoid Operation appeared first on Sparrow Computing.
Optimization Methods, Gradient Descent
This article covers a sublime explanation and a simple example of Vanilla Gradient Descent algorithm, Stochastic Gradient Descent, Momentum Optimizer, and Adam Optimizer in which RMSProp is also explainedHow Anaconda Supports the Cybersecurity Executive Order
This Executive Order is an essential step toward reversing this tide of malicious cyber campaigns that imperil information and operational technology across the country. At Anaconda, we are committed to doing our part by continuing to innovate tools that support our customers’ abilities to harness open-source software while maintaining the highest enterprise security standards. While vulnerabilities are inevitable, if we work together in the spirit of transparency and collaboration, breaches and hacks don’t have to be.Creating your first app with PySide — A simple Hello World! application with Python and Qt
PySide, also known as Qt for Python, is a Python library for creating GUI applications using the Qt toolkit. PySide is the official binding for Qt on Python and is now developed by The Qt Company itself.
There are two major versions available: PySide2 based on Qt5 and PySide6 based …
Understanding of Regularization in Neural Networks
This article includes the different techniques of regularization like Data Augmentation, L1, L2, Dropout, and Early StoppingDiving into Object Detection Basics
A guide for Object Detection basic concepts which cover What is Object Detection and how does it work, Concept of Anchor Boxes, Why is Loss function necessary, some free datasets, and finally, implementation of SSD.Normalization in Deep learning
Different types of Normalization in Deep Learning. A very useful technique to avoid overfitting and generalize your model better.Dropout in Deep Learning
Understanding Dropouts in Deep Learning to reduce overfittingYolov3 and Yolov4 in Object Detection
Explanation of object detection with various use cases and algorithms. Specifically, how the yolov3 and yolov4 architectures are structured, and how they perform object detectionEnd-To-End PyTorch Example of Image Classification with Convolutional Neural Networks
Image classification solutions in PyTorch with popular models like ResNet and its variations. End-To-End solution for CIFAR10/100 and ImageNet datasets.Supervised learning with Scikit-Learn Library
How to create a model for supervised learning like linear and logistic regression with scikit-learn python libraryLinear and Logistic Regression
Intuition and implementation behind the base algorithms for supervised machine learningRandom Forests Understanding
Intuition and Implementation on a key algorithm to reduce overfitting in tree based algorithmsActivation Functions for Neural Networks
In this article, explaination of various activation functions has been given like Linear, ELU, ReLU, Sigmoid, and tanh.Rethinking Jupyter Interactive Documentation
Jupyter Notebook first release was 8 years ago – under the IPython Notebook name at the time. Even if notebooks were not invented by Jupyter; they were definitely democratized by it. Being Web powered allowed development of many changes in the Datascience world. Objects now often expose rich representation; from Pandas dataframes with as html tables, to more recent Scikit-learn model.
Today I want to look into a topic that has not evolved much since, and I believe could use an upgrade. Accessing interactive Documentation when in a Jupyter session, and what it could become. At the end I'll link to my current prototype if you are adventurous.
Read more… (7 min remaining to read)
Anaconda Content Trust: Conda Signature Verification
Animations and Transformations with QtQuick — Building an animated analog clock in QML
In the previous tutorial we implemented a basic QML clock application using Python code to get the current time, format it into a string and send that through to our QML layout for display using Qt signals.
That gave us a good overview of the structure of Python/QML applications …
Data Scientists: Bring the Narrative to the Forefront
Charts and graphs can be a valuable tool, as mentioned earlier, but they can also carry a sense of authority that overshadows the reality that data visualizations or non-visual presentations can be manipulative (see the margarine-and-divorce-rates example above). This is because data is often seen as a fixed source of truth, yet the presentation of data is in fact subjective, largely dependent on who’s interpreting it and how. While data visualization can help identify trends, patterns, or outliers in datasets, it can rarely be used for full analysis and interpretation by itself.Spot the differences: what is new in Spyder 5?
In case you missed it, Spyder 5 was released at the beginning of April! This blog post is a conversation attempting to document the long and complex process of improving Spyder's UI with this release. Portions lead by Juanita Gomez are marked as Juanita, and those lead by Isabela Presedo-Floyd are marked as Isabela.
What did we do?[Juanita] Spyder was created more than 10 years ago and it has had the contributions of a great number of developers who have written code, proposed ideas, opened issues and tested PRs in order to build a piece of Spyder on their own. We (the Spyder team) have been lucky to have such a great community of people contributing throughout the years, but this is the first time that we decided to ask for help from an UX/UI expert! Why? You might wonder. Having the contributions of this great amount of people has resulted in inconsistencies around Spyder’s interface which we didn’t stop
On the Link Between Optimization and Polynomials, Part 4
While the most common accelerated methods like Polyak and Nesterov incorporate a momentum term, a little known fact is that simple gradient descent –no momentum– can achieve the same rate through only a well-chosen sequence of step-sizes. In this post we'll derive this method and through simulations discuss its practical …
A step towards educating with Spyder
As a community manager in the Spyder team, I have been looking for ways of involving more users in the community and making Spyder useful for a larger number of people. With this, a new idea came: Education.
For the past months, we have been wondering with the team whether Spyder could also serve as a teaching-learning platform, especially in this era where remote instruction has become necessary. We submitted a proposal to the Essential Open Source Software for Science (EOSS) program of the Chan Zuckerberg Initiative, during its third cycle, with the idea of providing a simple way inside Spyder to create and share interactive tutorials on topics relevant to scientific research. Unfortunately, we didn’t get this funding, but we didn’t let this great idea die.
We submitted a second proposal to the Python Software Foundation from which we were awarded $4000. For me, this is the perfect opportunity for us to take the first step towards using Spyder for education.
Read more… (2 min remaining to read)
NumFOCUS Welcomes Tesco Technology to Corporate Sponsors
NumFOCUS is pleased to announce our new partnership with Tesco Technology. A long-time PyData event sponsor, Tesco Technology joined NumFOCUS as a Silver Corporate Sponsor in December 2020. “We are very excited to formalize our partnership with Tesco Technology,” said Leah Silen, NumFOCUS Executive Director. “Tesco Technology has partnered with NumFOCUS for the past several […]
The post NumFOCUS Welcomes Tesco Technology to Corporate Sponsors appeared first on NumFOCUS.
PyQt6 Book now available: Create GUI Applications with Python & Qt6 — The hands-on guide to making apps with Python
Hello! Today I have released the first PyQt6 edition of my book Create GUI Applications, with Python & Qt6.
This update follows the 4th Qt5 Edition of my PyQt book updating all the code examples and adding additional PyQt6-specific detail. The book contains 600+ pages and 200+ complete code examples taking …
PyTorch TensorIterator Internals - 2021 Update
For contributors to the PyTorch codebase, one of the most commonly encountered
C++ classes is
TensorIterator.
TensorIterator offers a standardized way to iterate over elements of
a tensor, automatically parallelizing operations, while abstracting device and
data type details.
In April 2020, Sameer Deshmukh wrote a blog article discussing PyTorch TensorIterator Internals. Recently, however, the interface has changed significantly. This post describes how to use the current interface as of April 2021. Much of the information from the previous article is directly copied here, but with updated API calls and some extra details.
Read more… (8 min remaining to read)
Job Posting | Communications and Marketing Manager
Job Title: Communications and Marketing Manager Position Overview The primary role of the Communications & Marketing Manager is to manage the NumFOCUS brand by overseeing all outgoing communications between NumFOCUS and our stakeholders. You will serve the project communities by playing a key role in their event marketing management and assist with project promotional and […]
The post Job Posting | Communications and Marketing Manager appeared first on NumFOCUS.
There Is No Data – Only Frozen Models
For a deeper look at this topic, check out this episode from The a16z Podcast, featuring a conversation between Peter and Martin Casado.Create applications with QtQuick — Build modern applications with declarative QML
In previous tutorials we've used the Qt Widgets API for building our applications. This has been the standard method for building applications since Qt was first developed. However, Qt provides another API for building user interfaces: Qt Quick. This is a modern mobile-focused API for app development, with which you …
Getting started with Acoular - Part 1
This is the first in a series of three blog posts about the basic use of Acoular. It explains some fundamental concepts and walks through a simple example. Acoular is a Python library that processes multichannel data (up to a few hundred channels) from acoustic measurements with a microphone array. The focus of the processing is on the construction of a map of acoustic sources. This is somewhat similar to taking an acoustic photograph of some sound sources.Getting started with Acoular - Part 2
This is the second in a series of three blog posts about the basic use of Acoular. It assumes that you already have read the first post and continues by explaining some more concepts and additional methods. Acoular is a Python library that processes multichannel data (up to a few hundred channels) from acoustic measurements with a microphone array. The focus of the processing is on the construction of a map of acoustic sources. This is somewhat similar to taking an acoustic photograph of some sound sources.Getting started with Acoular - Part 3
This is the third and final in a series of three blog posts about the basic use of Acoular. It assumes that you already have read the first two posts and continues by explaining additional concepts to be used with time domain methods. Acoular is a Python library that processes multichannel data (up to a few hundred channels) from acoustic measurements with a microphone array. The focus of the processing is on the construction of a map of acoustic sources. This is somewhat similar to taking an acoustic photograph of some sound sources. To continue, we do the same set up as in Part 1. However, as we are setting out to do some signal processing in time domain, we define only TimeSamples, MicGeom, RectGrid and SteeringVector objects but no PowerSpectra or BeamformerBase. import acoular ts = acoular.TimeSamples( name="three_sources.h5" ) mg = acoular.MicGeom( from_file="array_64.xml" ) rg = acoular.RectGrid( x_min=-0.2, x_max=0.2, y_min=-0.2, y_max=0.2, z=0.3, increment=0.01 ) st = acoular.SteeringVector( grid=rg, mics=mg (continued...)Why Organizations Should Invest in a Chief Data Officer
Not every company may be ready for a dedicated Chief Data Scientist; some may need to roll these particular responsibilities into a broader CDO position, while others might need to set aside a percentage of a CIO or CTO’s time to devote to data issues. But every company should have at least one senior leader who is accountable for ensuring the organization gets strategic value from its data and stewards it ethically and legally. Data science and ML can unlock vast benefits for organizations, but only if these initiatives are handled responsibly and thoughtfully. This is why the trend toward an increasing number of C-level data roles will continue.Accessibility: Who's Responsible?
For the past few months, I've been part of a group of people in the JupyterLab community who've committed to start chipping away at the many accessibility failings of JupyterLab. I find this work is critical, fascinating, and a learning experience for everyone involved. So I'm going to document my personal experience and lessons I've learned in a series of blog posts. Welcome!
Read more… (6 min remaining to read)
Geophysics in the Cloud Competition
Join the 2021 GSH Geophysics in the cloud competition. Build a novel seismic inversion app and access all the data on demand with serverless cloud storage. Example notebooks show how to access this data and use AWS SageMaker to build your ML models. With prizes. Author: Ben Lasscock, Ph.D., Manager, Strategic Technologies, Energy Solutions …Continue Reading
PyTorch Tensor to NumPy Array and Back
You can easily convert a NumPy array to a PyTorch tensor and a PyTorch tensor to a NumPy array. This post explains how it works.
The post PyTorch Tensor to NumPy Array and Back appeared first on Sparrow Computing.
TorchVision Transforms: Image Preprocessing in PyTorch
TorchVision, a PyTorch computer vision package, has a great API for image pre-processing in its torchvision.transforms module. This post gives some basic usage examples, describes the API and shows you how to create and use custom image transforms.
The post TorchVision Transforms: Image Preprocessing in PyTorch appeared first on Sparrow Computing.
PySide6 Book now available: Create GUI Applications with Python & Qt6 — The hands-on guide to making apps with Python
Hello! This morning I released the first Qt6 edition of my PySide book Create GUI Applications, with Python & Qt6.
This update follows the 4th Edition of the PySide book updating all the code examples and adding additional PySide6-specific detail. The book contains 600+ pages and 200+ complete code examples taking …
PyQt6 vs PySide6: What's the difference between the two Python Qt libraries? — ...and what's exactly the same (most of it)
There is a new version of Qt (version 6) and with it new versions of PyQt and PySide -- now named PyQt6 & PySide6 respectively. In preparation for the Qt6 editions of my PyQt5 & PySide2 books I've been looking at the latest versions of the libraries to identify the differences between them …
Giving Visibility to Renewable Energy
The EnergizAIR Infrastructure framework and key interfaces, with the Enthought responsibility on the project shown in the central, grey box. The ultimate project goal was to raise individual awareness of the contribution of renewable energy sources, and ultimately change behaviors. Now ten years later, with orders of magnitude more data, AI/machine learning, cloud, and smartphones …Continue Reading
sourmash 4.0 is now available! Low low cost if you buy now!
sourmash v4.0.0 is here!
NumPy Where: Understanding np.where()
The NumPy where function is like a vectorized switch that you can use to combine two arrays.
The post NumPy Where: Understanding np.where() appeared first on Sparrow Computing.
Finding the Mode of an Empirical Continuous Distribution
You can find the mode of an empirical continuous distribution by plotting the histogram and looking for the maximum bin.
The post Finding the Mode of an Empirical Continuous Distribution appeared first on Sparrow Computing.
On the Link Between Optimization and Polynomials, Part 3
I've seen things you people wouldn't believe.
Valleys sculpted by trigonometric functions.
Rates on fire off the shoulder of divergence.
Beams glitter in the dark near the Polyak gate.
All those landscapes will be lost in time, like tears in rain.
Time to halt.
A momentum optimizer *
PDF Report generator — Generate custom PDF reports using reportlab & pdfrw
If your job involves generating PDF reports, invoices, etc. you have probably thought about automating that with Python. Python has some great libraries for working with PDF files, allowing you to read and write PDFs from scripts. But you can also use these libraries as the basic of simple GUI …
NumPy All: Understanding np.all()
The np.all() function tests whether all elements in a NumPy array evaluate to true.
The post NumPy All: Understanding np.all() appeared first on Sparrow Computing.
Enhancements to Numba's guvectorize decorator
Starting from Numba 0.53, Numba will ship with an enhanced version of the @guvectorize decorator. Similar to the @vectorize decorator, @guvectorize now has two modes of operation:
- Eager, or decoration-time compilation and
- Lazy, or call-time compilation
Before, only the eager approach was supported. In this mode, users are required to provide a list of concrete supported types beforehand as its first argument. Now, this list can be omitted if desired and as one calls it, Numba dynamically generates new kernels for previously unsupported types.
Read more… (3 min remaining to read)
Introducing PyMC Labs: Saving the World with Bayesian Modeling
After I left Quantopian in 2020, something interesting happened: various companies contacted me inquiring about consulting to help them with their PyMC3 models.
Usually, I don't hear how people are using PyMC3 -- they mostly show up on GitHub or Discourse when something isn't working right. So, hearing about all these …
Binary Cross Entropy Explained
A simple NumPy implementation of the binary cross entropy loss function and some intuition about why it works.
The post Binary Cross Entropy Explained appeared first on Sparrow Computing.
Filtering DataFrames with the .query() Method in Pandas
Pandas provides a .query() method on DataFrame's with a convenient string syntax for filtering DataFrames. This post describes the method and gives simple usage examples.
The post Filtering DataFrames with the .query() Method in Pandas appeared first on Sparrow Computing.
sourmash v4.0.0 release candidate 1 is now available for comment!
sourmash v4.0.0 is coming!
SciPy 2021
As in 2020, this year’s SciPy Conference will be virtual, offering increased opportunities for attendance. 2020 set an attendance record of over 1,500, almost double the 2019 Austin, Texas conference. The event brings together attendees from industry, academia, national labs and more – showcasing projects, sharing knowledge and collaborating on code development. Author: Kristen Leiser, …Continue Reading
Linear Interpolation in Python: An np.interp() Example
It's easy to linearly interpolate a 1-dimensional set of points in Python using the np.interp() function from NumPy.
The post Linear Interpolation in Python: An np.interp() Example appeared first on Sparrow Computing.
Job Posting | Events and Digital Marketing Coordinator
Job Title: Events and Digital Marketing Coordinator Position Overview The primary role of the Events and Digital Marketing Coordinator is to support and assist the Events Manager and the Community Communications and Marketing Manager to advance one of NumFOCUS’s primary missions of educating and building the community of users and developers of open source scientific […]
The post Job Posting | Events and Digital Marketing Coordinator appeared first on NumFOCUS.
Strategy, Digitalization and Global Trends – C Suite Reflections
Key points are presented from the first of a series of LinkedIn articles where JSR Board Chairman Mitsunobu Koshiba (‘Nobu’) provides thought provoking insights on business strategy in the context of trends in three time horizons. The short term is dominated by an increased acceptance of Modern Monetary Theory. The mid-term is a shift in …Continue Reading
Transition your Python project to use pyproject.toml and setup.cfg! (An example.)
Updating old Python packages, in this year of the PSF 2021!
A snakemake hack for checkpoints
snakemake checkpoints r awesome
Python packaging in 2021 - pain points and bright spots
At Quansight we have a weekly "Q-share" session on Fridays where everyone can share/demo things they have worked on, recently learned, or that simply seem interesting to share with their colleagues. This can be about anything, from new utilities to low-level performance, from building inclusive communities to how to write better documentation, from UX design to what legal & accounting does to support the business. This week I decided to try something different: hold a brainstorm on the state of Python packaging today.
The ~30 participants were mostly from the PyData world, but not exclusively - it included people with backgrounds and preferences ranging from C, C++ and Fortran to JavaScript, R and DevOps - and with experience as end-users, packagers, library authors, and educators. This blog post contains the raw output of the 30-minute brainstorm (only cleaned up for textual issues) and my annotations on it (in italics) which capture some of the discussion during the session and links and context that may be helpful. I think it sketches a decent picture of
Making SciPy's Image Interpolation Consistent and Well Documented
SciPy's ndimage module provides a powerful set of general, n-dimensional image processing operations, categorized into areas such as filtering, interpolation and morphology. Traditional image processing deals with 2D arrays of pixels, possibly with an additional array dimension of size 3 or 4 to represent color channel and transparency information. However, there are many scientific applications where we may want to work with more general arrays such as the 3D volumetric images produced by medical imaging methods like computed tomography (CT) or magnetic resonance imaging (MRI) or biological imaging approaches such as light sheet microscopy. Aside from spatial axes, such data may have additional axes representing other quantities such as time, color, spectral frequency or different contrasts. Functions in ndimage have been implemented in a general n-dimensional manner so that they can be applied across 2D, 3D or more dimensions. A more detailed overview of the module is available in the SciPy ndimage tutorial. SciPy's image functions are
Run SAS in Python without Installation
Do you wish
(continued...)Welcoming Tania Allard as Quansight Labs co-director
Today I'm incredibly excited to welcome Tania Allard to Quansight as Co-Director of Quansight Labs. Tania (GitHub, Twitter, personal site) is a well-known and prolific PyData community member. In the past few years she has been involved as a conference organizer (JupyterCon, SciPy, PyJamas, PyCon UK, PyCon LatAm, JuliaCon and more), as a community builder (PyLadies, NumFOCUS, RForwards), as a contributor to Matplotlib and Jupyter, and as a regular speaker and mentor. She also brings relevant experience in both industry and academia - she joins us from Microsoft where she was a senior developer advocate, and has a PhD in computational modelling.
Read more… (4 min remaining to read)
Disnatia X/Potências de X
Nenhuma equipe de heróis me é tão querida quanto X-Men. Lá pelo final dos anos 90 comecei a colecionar por alguns anos, mas em seguida veio o fatídico aumento de preço com as Super-Heróis Premium, o que me acabou desmotivando a comprar. De lá para cá, acompanho esporadicamente, lendo notícias sobre, comprando uma ou outra… Continue a ler »Disnatia X/Potências de XWish Christmas with Python and R
This post is dedicated to all the Python and R Programming Lovers...Flaunt your knowledge in your peer group with the following programs. As a data science professional, you want your wish to be special on eve of christmas. If you observe the code, you may also learn 1-2 tricks which you can use later in your daily tasks.
Method 1 : Run the following program and see what I mean
R Code
paste(intToUtf8(acos(log(1))*180/pi-13),
toupper(substr(month.name[2],2,2)),
paste(rep(intToUtf8(acos(exp(0)/2)*180/pi+2^4+3*2),2), collapse = intToUtf8(0)),
LETTERS[5^(3-1)], intToUtf8(atan(1/sqrt(3))*180/pi+2),
toupper(substr(month.abb[10],2,2)),
intToUtf8(acos(log(1))*180/pi-(2*3^2)),
toupper(substr(month.name[4],3,4)),
intToUtf8(acos(exp(0)/2)*180/pi+2^4+3*2+1),
intToUtf8(acos(exp(0)/2)*180/pi+2^4+2*4),
intToUtf8(acos(log(1))*180/pi-13),
LETTERS[median(0:2)],
intToUtf8(atan(1/sqrt(3))*180/pi*3-7),
sep = intToUtf8(0)
)
Python Code
import math
import datetime
(chr(int(math.acos(math.log(1))*180/math.pi-13)) \
+ datetime.date(1900, 2, 1).strftime('%B')[1] \
+ 2 * datetime.date(1900, 2, 1).strftime('%B')[3] \
+ datetime.date(1900, 2, 1).strftime('%B')[7] \
+ chr(int(math.atan(1/math.sqrt(3))*180/math.pi+2)) \
+ datetime.date(1900, 10, 1).strftime('%B')[1] \
+ chr(int(math.acos(math.log(1))*180/math.pi-18)) \
+ datetime.date(1900, 4, 1).strftime('%B')[2:4] \
+ chr(int(math.acos(math.exp(0)/2)*180/math.pi+2**4+3*2+1)) \
+ chr(int(math.acos(math.exp(0)/2)*180/math.pi+2**4+2*4)) \
+ chr(int(math.acos(math.log(1))*180/math.pi-13)) \
+ "{:c}".format(97) \
+ chr(int(math.atan(1/math.sqrt(3))*180/math.pi*3-7))).upper()
Turn on computer speakers before running the code.
R Code
install.packages("audio")
library(audio)
christmas_file <- tempfile()
download.file("https://sites.google.com/site/pocketecoworld/merrychristmas1.wav", christmas_file, mode = "wb")
xmas
On the Link Between Optimization and Polynomials, Part 2
We can tighten the analysis of momentum methods through Chebyshev polynomials of the first and second kind. Following this connection, we'll derive one of the most iconic methods in optimization: Polyak momentum.