Add/update cocoapods and pypi download_url (and other) support to fetchcode/package.py

In connection with a purl2url issue in packageurl-python, we cannot add download URL support for cocoapods or pypi in purl2url.py because the process involves network calls, and consequently will need to add/update that support in fetchcode/package.py.

@pombredanne

@pombredanne I have a rather detailed question for you re metadata queries (i.e., using fetchcode/package.py) for a PURL with no version. The issue: missing podspec.json files, which we use to populate the metadata object for each version.

Sometimes the default URL we use, the raw.githubusercontent.com/CocoaPods/Specs... URL from cocoapods.py (with /blob deleted), returns a 404 for one or more of the tags I retrieve for that PURL from its GitHub repo. I currently print a statement to that effect in the console and add it to a new .log file in package.py, which I can access from the metadata command in purlcli.py and include in the headers/errors section of the metadata JSON output. Simple and clean, but can result in cocoapod tags for which we report no data.

SCTK's packagedcode/cocoapods.py also has a "backup" podspec URL: https://cdn.cocoapods.org/Specs/{hashed_path}/{name}/{version}/{name}.podspec.json. I don't currently use this as a backup in my code but could. FWIW, using sqlite as an example, both the default and the backup URLs return a 404 for the tags I've tried. BTW, these are the tags for sqlite: ['0.1.23', '0.1.22', '0.1.20', '0.1.18', '0.1.16', '0.1.15', '0.1.14', '0.1.13', '0.1.12', '0.1.11', '0.1.10', '0.1.9', '0.1.8', '0.1.7', '0.1.6', '0.1.5', '0.1.4', '0.1.3', '0.1.2', '0.1.1', '0.1.0'].

Finally, a 3rd podspec URL we might have available: sometimes, the cocoapods.org page has a podspec link to a location in the pod's GitHub repo. sqlite is an example: https://github.com/CocoaPods/Specs/blob/master/Specs/4/9/7/SQLite/0.1.23/SQLite.podspec.json. And sometimes that GitHub URL will also return a podspec.json for some of the other tags for that pod -- but not always.

How would you like this search, analysis and reporting process to be handled? For each tag, exhaust all 3 URLs if needed in case one of them has a podspec.json for the tag in the list, and do that for each tag in the list of tags from GitHub (which can be numerous)?

Note that the 3rd podspec URL example uses a different hashed_path from that used in the 2 cocoapods.py podspec URLs, e.g.

I don't (yet) know how that latter hashed_path is calculated....

^ It's the familiar MD5 hash we already use, applied to a cocoapod name that does not reflect the "official" uppercase/lowercase name as it exists in the cocoapods.org page for that pod, e.g., incorrect upper/lowercase of the input PURL in the PURL CLI metadata command (which of course leads to a different MD5 result ;-)

We now use the cocoapods.org case structure for the hashing and (via the .log file I've added to fetchcode/package.py) disclose that in the headers/warnings section of the JSON output (e.g., "Input PURL name 'afnetworking' processed as 'AFNetworking' per https://cocoapods.org/pods/afnetworking").