Planet SciPy

15 Best Tools for Tracking Machine Learning Experiments

15 Best Tools for Tracking Machine Learning Experiments While working on a machine learning project, getting good results from a single model-training run is one thing, but keeping all of your machine learning experiments organized and having a process that lets you draw valid conclusions from them is quite another. That’s what machine learning experiment […]

The post 15 Best Tools for Tracking Machine Learning Experiments appeared first on neptune.ai.

Two talks at JGI in May: sourmash, spacegraphcats, and disease associations in the human microbiome.

Using k-mers and taxonomy to find contamination in metagenomes

Creating the ultimate terminal experience in Spyder 4 with Spyder-Terminal

6 Reasons Your Open-Source Data Science Pipeline Needs Attention Now

Your enterprise data scientists are almost certainly using Anaconda Distribution alongside 20 million other practitioners worldwide. The Anaconda Distribution is a package and environment manager designed for solo data scientists, and it is the most…

The post 6 Reasons Your Open-Source Data Science Pipeline Needs Attention Now appeared first on Anaconda.

We’ve Reached a Milestone: pandas 1.0 Is Here

Today the pandas project announced the release of pandas 1.0.0. For more on what’s changed, read through the extensive release notes. We’re particularly excited about Numba-accelerated window operations and the new nullable boolean and string…

The post We’ve Reached a Milestone: pandas 1.0 Is Here appeared first on Anaconda.

Introducing Anaconda Team Edition: Secure Open-Source Data Science for the Enterprise

I’m very excited to announce a new addition to Anaconda’s product line — Anaconda Team Edition! For the last few years, Anaconda has offered two products: our free Anaconda Distribution, meant for individual practitioners, and…

The post Introducing Anaconda Team Edition: Secure Open-Source Data Science for the Enterprise appeared first on Anaconda.

Keras Metrics: Everything You Need To Know

Keras Metrics: Everything You Need To Know Keras metrics are functions that are used to evaluate the performance of your deep learning model. Choosing a good metric for your problem is usually a difficult task. you need to understand which metrics are already available in Keras and tf.keras and how to use them, in many […]

The post Keras Metrics: Everything You Need To Know appeared first on neptune.ai.

Advancing research software in the UK through an SSI fellowship

I have selected as part of the 2020 cohort of Fellows of the Software Sustainability Institute!

The Institute cultivates world-class research with software. It's based at the universities of Edinburgh, Manchester, Southampton, and Oxford in the UK. Their motto says it all:

The SSI has a yearly fellowship program to fund the organization of communities around scientific software (creating of local user groups, workshops, hackathons, etc). Even more importantly, they organize several events to get current and past fellows in the same place doing awesome stuff. I'm really looking forward to this year's Collaborations Workshop (registration is open to all, not just fellows). I applied at the end of last year and was selected to join the 2020 cohort of fellows along with some truly amazing people.

My plan for the fellowship is to run

Exploratory Data Analysis for Natural Language Processing: A Complete Guide to Python Tools

Exploratory Data Analysis for Natural Language Processing: A Complete Guide to Python Tools Exploratory data analysis is one of the most important parts of any machine learning workflow and Natural Language Processing is no different. But which tools you should choose to explore and visualize text data efficiently? In this article, we will discuss and implement […]

The post Exploratory Data Analysis for Natural Language Processing: A Complete Guide to Python Tools appeared first on neptune.ai.

Don't fund Software that doesn't exist

sourmash-oddify: a workflow for exploring contamination in metagenome-assembled genomes

Using k-mers and taxonomy to find contamination in metagenomes

Finding problematic bacterial/archaeal genomes using k-mers and taxonomy

Some things in Genbank look ...odd.

How does sourmash's lca classification routine compare with GTDB classifications?

GTDB databases again!

Sourmash LCA databases now available for the GTDB taxonomy

GTDB databases!

metadsl PyData talk

class Number(metadsl.Expression):
   @metadsl.expression
   def __add__(self, other: Number) -> Number:
       ...

   @metadsl.expression
   @classmethod
   def from_int(cls, i: int) -> Number:
       ...


@metadsl.rule
def add_zero(y: Number):
   yield Number.from_int(0) + y, y
   yield y + Number.from_int(0), y

2019: A Year in Review

Before we dive into a new decade, we’re looking back on all we’ve accomplished together as a company and as a community in 2019. We’re excited to be part of such a vibrant and growing…

The post 2019: A Year in Review appeared first on Anaconda.

8 AI Predictions for 2020: Business Leaders & Researchers Weigh In

The first industrial revolution was powered by coal, the second by oil and gas, and the third by nuclear power. The fourth — AI — is fueled by an abundance of data and breakthroughs in…

The post 8 AI Predictions for 2020: Business Leaders & Researchers Weigh In appeared first on Anaconda.

It’s Time to Upgrade to Python 3 – Time Is Running Out!

The Python community will sunset Python 2 on January 1, 2020.   How much time is left? Check out the handy clock at: https://pythonclock.org/.  How does this impact the Anaconda Distribution and the conda packages Anaconda…

The post It’s Time to Upgrade to Python 3 – Time Is Running Out! appeared first on Anaconda.

Two PhD studentships at the University of Liverpool

I have two open positions for funded studentships at the University of Liverpool. Applications are open until 10 January 2020.

Follow the links for more detailed versions.

Bringing machine learning techniques to geophysical data processing

The goal of this project is to investigate the use of existing machine learning techniques to process gravity and magnetics data using the Equivalent Layer Method. The methods and software developed during this project can be applied to process large amounts of gravity and magnetics data, including airborne and satellite surveys, and produce data products that can enable further scientific investigations. Examples of such data products include global gravity gradient grids from GOCE satellite measurements, regional magnetic grids for the UK, gravity grids for the Moon and Mars, etc.

Large-scale mapping of the thickness of the

How Machine Learning Will Generate up to $2 Trillion in Value for the Manufacturing Industry

Today manufacturers must create high-quality products as efficiently and consistently as possible to remain competitive. With the increasing price of raw materials, skilled labor shortages, and rising competition, this can be difficult. Fortunately, machine learning…

The post How Machine Learning Will Generate up to $2 Trillion in Value for the Manufacturing Industry appeared first on Anaconda.

SunPy Receives NASA Grant, Helps Generate Parker Solar Probe Results

The post SunPy Receives NASA Grant, Helps Generate Parker Solar Probe Results appeared first on NumFOCUS.

NumFOCUS Kicks Off Year-End Fundraising with $2,500 Gift

Giving Tuesday 2019 marked the start of NumFOCUS’s year-end fundraising campaign, and this year’s effort began with a major donation from the organization’s Board President, Andy Terrel. “Today I’m pledging a gift of $2,500 to help kick off the NumFOCUS end-of-year fundraising campaign,” Terrel wrote yesterday in an e-mail to the NumFOCUS community. “I believe […]

The post NumFOCUS Kicks Off Year-End Fundraising with $2,500 Gift appeared first on NumFOCUS.

Stepping Onto a New Path…

The post Stepping Onto a New Path… appeared first on NumFOCUS.

Get Python Package Download Statistics with Condastats

Hundreds of millions of Python packages are downloaded using Conda every month. That’s why we are excited to announce the release of condastats, a conda statistics API with Python interface and Command Line interface. Now anyone…

The post Get Python Package Download Statistics with Condastats appeared first on Anaconda.

Getting a big scientific prize for open-source software

A few days ago, Loïc Estève, Alexandre Gramfort, Olivier Grisel, Bertrand Thirion, and myself received the “Académie des Sciences Inria prize for transfer”, for our contributions to the scikit-learn project …

Variable Explorer improvements in Spyder 4

Variable Explorer improvements in Spyder 4

This blogpost was originally published on the Quansight Labs website.

Spyder 4 will be released very soon with lots of interesting new features that you'll want to check out, reflecting years of effort by the team to improve the user experience. In this post, we will be talking about the improvements made to the Variable Explorer.

These include the brand new Object Explorer for inspecting arbitrary Python variables, full support for MultiIndex dataframes with multiple dimensions, and the ability to filter and search for variables by name and type, and much more.

It is important to mention that several of the above improvements were made possible through integrating the work of two other projects. Code from gtabview was used to implement the multi-dimensional Pandas indexes, while objbrowser was the foundation of the new Object Explorer.

For Spyder 4 we added a long-requested feature: full support for inspecting any kind of Python object through the Variable

Astropy Receives $900k Grant from Moore Foundation

The post Astropy Receives $900k Grant from Moore Foundation appeared first on NumFOCUS.

mlpack Machine Learning Library joins NumFOCUS Sponsored Projects

The post mlpack Machine Learning Library joins NumFOCUS Sponsored Projects appeared first on NumFOCUS.

Now Hiring: Matplotlib Research Software Engineering Fellow

The post Now Hiring: Matplotlib Research Software Engineering Fellow appeared first on NumFOCUS.

NumFOCUS Summit 2019

The post NumFOCUS Summit 2019 appeared first on NumFOCUS.

2019 NumFOCUS Awards and New Contributor Recognition

The post 2019 NumFOCUS Awards and New Contributor Recognition appeared first on NumFOCUS.

A new grant for NumPy and OpenBLAS!

Chan Zuckerberg Initiative Funds Maintenance of NumFOCUS Projects

The post Chan Zuckerberg Initiative Funds Maintenance of NumFOCUS Projects appeared first on NumFOCUS.

Essential Open-Source Library pandas Awarded CZI Grant to Further Development

We’re pleased to announce that pandas, the open-source library providing high-performance data structures for tabular data analysis, has received grant funding from the Chan Zuckerberg Initiative (CZI) as part of their Essential Open Source Software…

The post Essential Open-Source Library pandas Awarded CZI Grant to Further Development appeared first on Anaconda.

Optuna vs Hyperopt: Which Hyperparameter Optimization Library Should You Choose?

Optuna vs Hyperopt: Which Hyperparameter Optimization Library Should You Choose? Thinking which library should you choose for hyperparameter optimization? Been using Hyperopt for a while and feel like changing? Just heard about Optuna and you want to see how it works? Good! In this article I will: show you an example of using Optuna and […]

The post Optuna vs Hyperopt: Which Hyperparameter Optimization Library Should You Choose? appeared first on neptune.ai.

File management improvements in Spyder4

File management improvements in Spyder 4

This blogpost was originally published on the Quansight Labs website.

Version 4.0 of Spyder is almost ready! It has been in the making for well over two years, and it contains lots of interesting new features. We will focus on the Files pane in this post, where we've made several improvements to the interface and file management tools.

In order to simplify the Files pane's interface, the columns corresponding to size and kind are hidden by default. To change which columns are shown, use the top-right pane menu or right-click the header directly.

First, we added the ability to associate different external applications with specific file extensions they can open. Under the File associations tab of the Files preferences pane, you can add file types and set the external program used to open each of them by default.

Once you've set this up, files will automatically launch in the associated application when opened from the Files pane in Spyder.

uarray: Attempting to move the ecosystem forward

How to Do Hyperparameter Tuning on Any Python Script in 3 Easy Steps

How to Do Hyperparameter Tuning on Any Python Script in 3 Easy Steps You wrote a Python script that trains and evaluates your machine learning model. Now, you would like to automatically tune hyperparameters to improve its performance? I got you! In this article, I will show you how to convert your script into an […]

The post How to Do Hyperparameter Tuning on Any Python Script in 3 Easy Steps appeared first on neptune.ai.

Scikit-Optimize: Bayesian Hyperparameter Optimization in Python

Scikit Optimize: Bayesian Hyperparameter Optimization in Python So you want to optimize hyperparameters of your machine learning model and you are thinking whether Scikit Optimize is the right tool for you? You are in the right place. In this article I will: show you an example of using skopt on a real problem, evaluate this […]

The post Scikit-Optimize: Bayesian Hyperparameter Optimization in Python appeared first on neptune.ai.

Experiment Management: How to Organize Your Model Development Process

Experiment Management: How to Organize Your Model Development Process Let me share a story that I’ve heard too many times. ” So I was developing a machine learning model with my team and within a few weeks of extensive experimentation we got promising results… …unfortunately, we couldn’t tell exactly what performed best because we didn’t […]

The post Experiment Management: How to Organize Your Model Development Process appeared first on neptune.ai.

F1 Score vs ROC AUC vs Accuracy vs PR AUC: Which Evaluation Metric Should You Choose?

F1 Score vs ROC AUC vs Accuracy vs PR AUC: Which Evaluation Metric Should You Choose? So you are working on a machine learning project and thinking: When accuracy is a better evaluation metric than ROC AUC? When should I not use accuracy? What is the F1 score good for? If my problem is highly […]

The post F1 Score vs ROC AUC vs Accuracy vs PR AUC: Which Evaluation Metric Should You Choose? appeared first on neptune.ai.

The Austin American-Statesman Names Anaconda a Winner of the Austin Top Workplaces 2019 Award

Anaconda is thrilled to have been awarded a Top Workplaces 2019 honor by The Austin American-Statesman. The list of winners is based solely on employee feedback gathered through a third-party survey administered by research partner Energage,…

The post The Austin American-Statesman Names Anaconda a Winner of the Austin Top Workplaces 2019 Award appeared first on Anaconda.

This Week in Glean: November 1, 2019

When data goes wrong

Loan Amortisation Schedule using R and Python

24 Evaluation Metrics for Binary Classification (And When to Use Them)

24 Evaluation Metrics for Binary Classification (And When to Use Them) Not sure which evaluation metric you should choose for your binary classification problem? After reading this blog post you should have a good idea. You will learn about a bunch of common and lesser-known evaluation metrics and charts to understand how to choose the […]

The post 24 Evaluation Metrics for Binary Classification (And When to Use Them) appeared first on neptune.ai.

Introducing napari: a fast n-dimensional image viewer in Python

NumFOCUS and Tidelift partner to support essential community-led open source data science and scientific computing projects

The post NumFOCUS and Tidelift partner to support essential community-led open source data science and scientific computing projects appeared first on NumFOCUS.

What Can AI Teach Us about Bias and Fairness?

By: Peter Wang & Natalie Parra-Novosad As researchers, journalists, and many others have discovered, machine learning algorithms can deliver biased results. One notorious example is ProPublica’s discovery of bias in a software called COMPAS used…

The post What Can AI Teach Us about Bias and Fairness? appeared first on Anaconda.

Announcing Anaconda Distribution 2019.10

We are pleased to announce the release of Anaconda Distribution 2019.10! As there were some significant changes in the previous Anaconda Distribution 2019.07 installers, this release focuses on polishing up rough edges in that release…

The post Announcing Anaconda Distribution 2019.10 appeared first on Anaconda.

O SERPRO e a validação de documentos digitais

Quansight Labs Work Update for September, 2019

Os documentos digitais (no plural) do Brasil

How to Evaluate the Logistic Loss and not NaN trying

A naive implementation of the logistic regression loss can results in numerical indeterminacy even for moderate values. This post takes a closer look into the source of these instabilities and discusses more robust Python implementations.

Uvas Gold 200

My poem about a rainy 200k was published in the Fall 2019 issue of American Randonneur (a quarterly magazine published by Randonneurs USA)

I've been doing samizdat poetry for as long as I've had a web presence (since 1999), but I am now officially a published poet! (I am deliberately not counting the embarrasing hackjob that was published in a youth anthology when I was in 8th grade.)

You can find "Uvas Gold 200" on page 26 - either directly on this skeuomorphic leafing viewer or the PDF, but I'm republishing both the exposition blurb and the poem below. If you prefer to listen, I recorded a reading of it that you can download in different flavors: a local audio only, a local video, or the embeded video version below.

Uvas Gold 200k starts and ends in Fremont, CA and was held on Saturday, December 1st, 2018. The ride frontloads the climbing by going nearly half-way up Mount

Introducing Our Newest Corporate Sponsorship Prospectus

The post Introducing Our Newest Corporate Sponsorship Prospectus appeared first on NumFOCUS.

Grupo de Estudos do Laboratório Amazônico de Estudos Sociotécnicos – UFPA

Ruby wrappers for the XND project

Highlights From The 2019 Pandas Hack

The post Highlights From The 2019 Pandas Hack appeared first on NumFOCUS.

Quansight Labs Dask Update

pelas ruas de Belém

Spyder 4.0 beta4: Kite integration is here

Spyder 4.0: Kite integration is here

This blogpost was originally published on the Quansight Labs website.

Note: Kite is sponsoring the work discussed in this blog post, and in addition supports Spyder 4.0 development through a Quansight Labs Community Work Order.

As part of our next release, we are proud to announce an additional completion client for Spyder, Kite. Kite is a novel completion client that uses Machine Learning techniques to find and predict the best autocompletion for a given text. Additionally, it collects improved documentation for compiled packages, e.g. Matplotlib, NumPy and SciPy, that cannot be obtained easily by using traditional code analysis packages such as Jedi. Although Kite is not open source like Spyder, you can download it without charge at the Kite website.

By incorporating Kite into Spyder, we will improve and provide the ultimate autocompletion and signature retrieval experience for most of the scientific Python stack and beyond. For instance, let’s take a look at the following PyTorch completion. While

An initial report on the Common Fund Data Ecosystem

Helping DCCs get FAIRer

Object Oriented Programming in Python : Learn by Examples

Precision Recall Curve Simplified

Comparing two genome binnings quickly with sourmash

Comparing two sets of MAGs, for fun and profit!

Calculate KS Statistic with Python

A Complete Guide to Python DateTime Functions

In this tutorial, we will cover python datetime module and how it is used to handle date, time and datetime formatted columns (variables). It includes various practical examples which would help you to gain confidence in dealing dates and times with python functions. In general, Date types columns are not easy to manipulate as it comes with a lot of challenges like dealing with leap years, different number of days in a month, different date and time formats or if date values are stored in string (character) format etc.

Table of Contents

---

Introduction : datetime moduleIt is a python module which provides several functions for dealing with dates and time. It has four classes as follows which are explained in the latter part of this article how these classes work.

1. datetime
2. date
3. time
4. timedelta

People who have no experience of working with real-world datasets might have not encountered date columns. They might be under impression that working with dates is rarely used and not so

What are *args and **kwargs and How to use them

Python : 10 Ways to Filter Pandas DataFrame

Python Dictionary Comprehension with Examples

d = {'a': [1,2], 'b': [3,4], 'c': [5,6]}

d.keys() # 'a', 'b', 'c'
d.values() # [1, 2], [3, 4], [5,

HTML outputs in Jupyter

User interaction in data science projects can be improved by adding a small amount of visual deisgn.

To motivate effort around visual design we show several simple-yet-useful examples. The code behind these examples is small and accessible to most Python developers, even if they don’t have much HTML experience.

This post in particular focuses on Jupyter’s ability to add HTML output to any object. This can either be full-fledged interactive widgets, or just rich static outputs like tables or diagrams. We hope that by showing examples here we will inspire some throughts in other projects.

This post was supported by replies to this tweet. The rest of this post is just examples.

I originally decided to write this post after reading another blogpost from the UK Met office, where they included the HTML output of their library Iris in a a blogpost

(work by Peter Killick, post by Theo McCaie)

The fact that the output provided by an interactive session is the same output that you would provide in a published result helps everyone. The interactive

Python list comprehension with Examples

Don't cite the No Free Lunch Theorem

15 ways to read CSV file with pandas

import pandas as pd

Matplotlib Tutorial : Learn with Examples in 3 hours

This tutorial outlines how to perform plotting and data visualization in python using Matplotlib library. The objective of this post is to get you familiar with the basics and advanced plotting functions of the library. It contains several examples which will give you hands-on experience in generating plots in python.

Table of Contents

---

What is Matplotlib?It is a powerful python library for creating graphics or charts. It takes care of all of your basic and advanced plotting requirements in Python. It took inspiration from MATLAB programming language and provides a similar MATLAB like interface for graphics. The beauty of this library is that it integrates well with pandas package which is used for data manipulation. With the combination of these two libraries, you can easily perform data wrangling along with visualization and get valuable insights out of data. Like ggplot2 library in R, matplotlib library is the grammar of graphics in Python and most used library for charts in Python.

Basics

Write Short Blogposts

I encourage my colleagues to write blogposts more frequently. This is for a few reasons:

1. It informs your broader community what you’re up to, and allows that community to communicate back to you quickly.

   You communicating to the community fosters a sense of collaboration, openness, and trust. You gain collaborators, build momentum behind your work, and curate a body of knowledge that early adopters can consume to become experts quickly.

   Getting feedback from your community helps you to course-correct early in your work, and stops you from wasting time in inefficient courses of action.

   You can only work for a long time without communicating if you are either entirely confident in what you’re doing, or reckless, or both.

2. It increases your visibility, and so is good for your career.

   I have a great job. I find my work to be both

How to encourage participation in teleconferences

Participation is good!

How to drop one or multiple columns in Pandas Dataframe

In this tutorial, we will cover how to drop or remove one or multiple columns from pandas dataframe.

What is pandas in Python?

pandas is a python package for data manipulation. It has several functions for the following data tasks:

1. Drop or Keep rows and columns
2. Aggregate data by one or more columns
3. Sort or reorder data
4. Merge or append multiple dataframes
5. String Functions to handle text data
6. DateTime Functions to handle date or time format columns

Import or Load Pandas library

To make use of any python library, we first need to load them up by using import command.

import pandas as pd
import numpy as np

Let's create a fake dataframe for illustration

The code below creates 4 columns named A through D.

df = pd.DataFrame(np.random.randn(6, 4), columns=list('ABCD'))

          A         B         C         D
0 -1.236438 -1.656038

String Functions in Python with Examples

This tutorial outlines various string (character) functions used in Python. To manipulate strings and character values, python has several in-built functions. It means you don't need to import or have dependency on any external package to deal with string data type in Python. It's one of the advantage of using Python over other data science tools. Dealing with string values is very common in real-world. Suppose you have customers' full name and you were asked by your manager to extract first and last name of customer. Or you want to fetch information of all the products that have code starting with 'QT'.

Table of Contents

---

List of frequently used string functions The table below shows many common string functions along with description and its equivalent function in MS Excel. We all use MS Excel in our workplace and familiar with the functions used in MS Excel. The comparison of string functions in MS EXCEL and Python would help you to learn

The cost of an open source contribution

TDK-Micronas partners with Quansight to sponsor Spyder

This blogpost was originally published on the Quansight Labs website

TDK-Micronas is sponsoring Spyder development efforts through Quansight Labs. This will enable the development of some features that have been requested by our users, as well as new features that will help TDK develop custom Spyder plugins in order to complement their Automatic Test Equipment (ATE’s) in the development of their Application Specific Integrated Circuits (ASIC’s).

At this point it may be useful to clarify the relationship the role of Quansight Labs in Spyder's development and the relationship with TDK. To quote Ralf Gommers (director of Quansight Labs):

"We're an R&D lab for open source development of core technologies around data science and scientific computing in Python. And focused on growing communities around those technologies. That's how I see it for Spyder as well: Quansight Labs enables developers to be employed to work on Spyder, and helps with connecting them to developers of other projects in similar situations. Labs should be an enabler to let the Spyder project, its community and individual developers grow.

Why citations are not enough for open source software

Spyder 4.0 takes a big step closer with the release of Beta 2!

This blogpost was originally published on the Quansight Labs website

It has been almost two months since I joined Quansight in April, to start working on Spyder maintenance and development. So far, it has been a very exciting and rewarding journey under the guidance of long time Spyder maintainer Carlos Córdoba. This is the first of a series of blog posts we will be writing to showcase updates on the development of Spyder, new planned features and news on the road to Spyder 4.0 and beyond.

First off, I would like to give a warm welcome to Edgar Margffoy, who recently joined Quansight and will be working with the Spyder team to take its development even further. Edgar has been a core Spyder developer for more than two years now, and we are very excited to have his (almost) full-time commitment to the project.

Since August 2018, when the first beta of the 4.x series was released, the Spyder development team has been

The Role of a Maintainer

What are the expectations and best practices for maintainers of open source software libraries? How can we do this better?

This post frames the discussion and then follows with best practices based on my personal experience and opinions. I make no claim that these are correct.

First, the most common answer to this question is the following:

• Q: What are expectations on OSS maintainers?
• A: Nothing at all. They’re volunteers.

However, let’s assume for a moment that these maintainers are paid to maintain the project some modest amount, like 10 hours a week.

How can they best spend this time?

Next, let’s disambiguate the role of developer, reviewer, and maintainer

1. Developers fix bugs and create features. They write code and docs and generally are agents of change in a software project. There are often many more developers than reviewers or maintainers.

2. Reviewers are known

Using GitHub for janky project reporting - some code

We scripted GitHub for lightweight project reporting

My first DNF (Ft Bragg 600k)

It's been six years since my first ride with The San Francisco Randonneurs and four years since my first 200k. I've ridden 18 rides that are at least that distance since then (3x 300k, 2x 400k, 1x 600k), completing my first Super Randonneur Series (2-, 3-, 4-, and 600k in one year) last year after not riding much the year before that. And this weekend I had my first DNF result on the Fort Bragg 600k. I Did Not Finish.

The best response to my choice of abandoning the ride to enjoy the campground came from Peter Curley, who said "That was a very mature decision." A clear departure from typical randonneuring stubbornness and refusal to give up, I celebrated my decision to quit as a victory when I arrived at the campground and made my announcement to the volunteers. I think I was so energetic about it that they did not believe me. I was being kind to myself, to my body, and at peace with the decision by

Should I Resign from My Full Professor Job to Work Fulltime on Cocalc?

PyCon2019 poem

I'm back in Cleveland for another Pycon. Yesterday was my first full day here. Along with Matt Seale, I was a helper at Matthias Bussonnier tutorial ("IPython and Jupyter in Depth: High productivity, interactive Python). The sticky system is efficient at signaling when someone in a classroom needs help, and a lot of folks don't know that this practice was popularized by Software Carpentry workshops and continues to be used at The Carpentries.

I stepped out for a coffee refill and bumped into a large contingent of Bloomberg folks I'd never met (Princeton office). I guess we have something like 90 people at the conference this year, and I made the usual and true remark about how I go to conferences to meet the other people who work at our company. Then after his tutorial concluded, Matthias and I bumped into Tracy Teal, exchanged some stickers, and chatted about The Carpentries, Jupyter, organizing conferences, governance and sponsorship models, and a bunch of other stuff.

Matthias was a

Why you should cite open source tools

Python Lambda Function with Examples

This article covers detailed explanation of lambda function of Python. You will learn how to use it in real-world data scenarios with examples.

Table of Contents

---

Introduction : Lambda FunctionIn non-technical language, lambda is an alternative way of defining function. You can define function inline using lambda. It means you can apply a function to some data using a single line of python code. It is called anonymous function as the function can be defined without its name. They are a part of functional programming style which focus on readability of code and avoids changing mutable data.

Syntax of Lambda Function

lambda arguments: expression

Lambda function can have more than one argument but expression cannot be more than 1. The expression is evaluated and returned. Example

addition = lambda x,y: x + y
addition(2,3) returns 5

In the above python code, x,y are the arguments and x + y is the expression that gets evaluated and returned.

Loops in Python explained with examples

Some questions and thoughts on journal peer review.

What's up with current peer review practice?

Create Dummy Data in Python

import pandas as pd
data = pd.DataFrame({"A" : ["John","Deep","Julia","Kate","Sandy"], 
                     "MonthSales" : [25,30,35,40,45]})

       A  MonthSales
0   John          25
1   Deep          30
2 

Things to think about when developing shotgun metagenome classifiers

Thoughts on goals and tradeoffs in classifying shotgun metagenome data.

Our submission for the NHGRI Human Genome Reference Center call

We wrote a big grant proposal!

PIP connection Error : SSL CERTIFICATE VERIFY FAILED

The most common issue in installing python package in a company's network is failure of verification of SSL Certificate. Sometimes company blocks some websites in their network so employees can't access these websites. Whenever they try to visit these websites, it shows "Access Denied because of company's policy". It causes connection error in reaching main python website.

Error looks like this :

Could not fetch URL https://pypi.python.org/: connection error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)

PIP SSL Certification Issue

Solution :

Run the following command. Make sure to specify package name in <package_name>

pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <package_name> -vvv

Suppose you want to install pandas package, you should submit the following line of command

pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org pandas -vvv

The --trusted-host option mark the host as trusted, even though it does not have valid or any HTTPS

Install Python Package

Python is one of the most popular programming language for data science and analytics. It is widely used for a variety of tasks in startups and many multi-national organizations. The beauty of this programming language is that it is open-source which means it is available for free and has very active community of developers across the world. Python developers share their solutions in the form of package or module with other python users. This tutorial explains various ways how to install python package.

Ways to Install Python Package

Method 1 : If Anaconda is already installed on your System

Anaconda is the data science platform which comes with pre-installed popular python packages and powerful IDE (Spyder) which has user-friendly interface to ease writing of python programming scripts.

If Anaconda is installed on your system (laptop), click on Anaconda Prompt as shown in the image below.

Anaconda Prompt

To install a python package or module, enter the code below in Anaconda Prompt -

pip install package-name

Install