JSON API for PyPi - how to list packages?

Question

There is a JSON API for PyPI which allows getting data for packages:

http://pypi.python.org/pypi/<package_name>/json
http://pypi.python.org/pypi/<package_name>/<version>/json

However, is it possible to get a list of all PyPI packages (or, for example, recent ones) with a GET call?

Answer 1

The easiest way to do this is to use the simple index at PyPI which lists all packages without overhead. You can then request the JSON of each package individually by performing a GET request to the URLs mentioned in your question.

Answer 2

I know that you asked for a way to do this from the JSON API, but you can use the XML-RPC api to get this info very easily, without having to parse HTML.

try:
     import xmlrpclib
except ImportError:
     import xmlrpc.client as xmlrpclib

client = xmlrpclib.ServerProxy('https://pypi.python.org/pypi')
# get a list of package names
packages = client.list_packages()

Answer 3

As of PEP 691, you can now grab this through the Simple API if you request a JSON response.

curl --header 'Accept: application/vnd.pypi.simple.v1+json' https://pypi.org/simple/ | jq

Answer 4

I tried this answer, but it's not working on Python 3.6

I found one solution with HTML parsing by using lxml package, But you have to install it via pip command as

pip install lxml

Then, try the following snippet

from lxml import html
import requests

response = requests.get("https://pypi.org/simple/")

tree = html.fromstring(response.content)

package_list = [package for package in tree.xpath('//a/text()')]

Answer 5

NOTE: To make tasks like this simple I've implemented an own Python module. It can be installed using pip:

pip install jk_pypiorgapi

The module is very simple to use. After instantiating an object representing the API interface you can make use of it:

import jk_pypiorgapi

api = jk_pypiorgapi.PyPiOrgAPI()
n = len(api.listAllPackages())
print("Number of packages on pypi.org:", n)

This module also provides capabilities for downloading information about specific packages as provided by pypi.org:

import jk_pypiorgapi
import jk_json

api = jk_pypiorgapi.PyPiOrgAPI()
jData = api.getPackageInfoJSON("jk_pypiorgapi")
jk_json.prettyPrint(jData)

This feature might be helpful as well.

Answer 6

This is now possible entirely within requests. The requested content type (the mime type for JSON) just needs to go into the dictionary of headers. `requests' can even decode the json into another dict for you:

r = requests.get(f'https://pypi.org/pypi/{package_name}/json', headers = {'Accept': 'application/json'});

info = r.json()['info']
print(f"requested package name = {package_name}, stored name: {info['name']}, author: {info['author']}, version: {info['version']}, license: {info['license']}")

Answer 7

Here's Bash one-liner:

curl -sG -H 'Host: pypi.org' -H 'Accept: application/json' https://pypi.org/pypi/numpy/json | awk -F "description\":\"" '{ print $2 }' |cut -d ',' -f 1

# NumPy is a general-purpose array-processing package designed to...