2

I have a normal organization repo list from curl response with command as below:

curl "https://api.github.com/orgs/[organization]/repos?access_token=[access_token]"

The output of this command is huge. I want to extract every repo's clone_url parameter, loop over those URLs and run git clone on it.

Below is the glimpse of one entry of tha JSON output curl gives me:

[
  {
    "id": 42059877,
    "name": "customrepo",
    "full_name": "SampleOrg/customrepo",
    "owner": {
      "login": "SampleOrg",
      "id": 14164701,
      "avatar_url": "https://avatars1.githubusercontent.com/u/14164701?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/SampleOrg",
      "html_url": "https://github.com/SampleOrg",
      "followers_url": "https://api.github.com/users/SampleOrg/followers",
      "following_url": "https://api.github.com/users/SampleOrg/following{/other_user}",
      "gists_url": "https://api.github.com/users/SampleOrg/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/SampleOrg/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/SampleOrg/subscriptions",
      "organizations_url": "https://api.github.com/users/SampleOrg/orgs",
      "repos_url": "https://api.github.com/users/SampleOrg/repos",
      "events_url": "https://api.github.com/users/SampleOrg/events{/privacy}",
      "received_events_url": "https://api.github.com/users/SampleOrg/received_events",
      "type": "Organization",
      "site_admin": false
    },
    "private": true,
    "html_url": "https://github.com/SampleOrg/customrepo",
    "description": null,
    "fork": false,
    "url": "https://api.github.com/repos/SampleOrg/customrepo",
    "forks_url": "https://api.github.com/repos/SampleOrg/customrepo/forks",
    "keys_url": "https://api.github.com/repos/SampleOrg/customrepo/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/SampleOrg/customrepo/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/SampleOrg/customrepo/teams",
    "hooks_url": "https://api.github.com/repos/SampleOrg/customrepo/hooks",
    "issue_events_url": "https://api.github.com/repos/SampleOrg/customrepo/issues/events{/number}",
    "events_url": "https://api.github.com/repos/SampleOrg/customrepo/events",
    "assignees_url": "https://api.github.com/repos/SampleOrg/customrepo/assignees{/user}",
    "branches_url": "https://api.github.com/repos/SampleOrg/customrepo/branches{/branch}",
    "tags_url": "https://api.github.com/repos/SampleOrg/customrepo/tags",
    "blobs_url": "https://api.github.com/repos/SampleOrg/customrepo/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/SampleOrg/customrepo/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/SampleOrg/customrepo/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/SampleOrg/customrepo/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/SampleOrg/customrepo/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/SampleOrg/customrepo/languages",
    "stargazers_url": "https://api.github.com/repos/SampleOrg/customrepo/stargazers",
    "contributors_url": "https://api.github.com/repos/SampleOrg/customrepo/contributors",
    "subscribers_url": "https://api.github.com/repos/SampleOrg/customrepo/subscribers",
    "subscription_url": "https://api.github.com/repos/SampleOrg/customrepo/subscription",
    "commits_url": "https://api.github.com/repos/SampleOrg/customrepo/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/SampleOrg/customrepo/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/SampleOrg/customrepo/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/SampleOrg/customrepo/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/SampleOrg/customrepo/contents/{+path}",
    "compare_url": "https://api.github.com/repos/SampleOrg/customrepo/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/SampleOrg/customrepo/merges",
    "archive_url": "https://api.github.com/repos/SampleOrg/customrepo/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/SampleOrg/customrepo/downloads",
    "issues_url": "https://api.github.com/repos/SampleOrg/customrepo/issues{/number}",
    "pulls_url": "https://api.github.com/repos/SampleOrg/customrepo/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/SampleOrg/customrepo/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/SampleOrg/customrepo/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/SampleOrg/customrepo/labels{/name}",
    "releases_url": "https://api.github.com/repos/SampleOrg/customrepo/releases{/id}",
    "deployments_url": "https://api.github.com/repos/SampleOrg/customrepo/deployments",
    "created_at": "2015-09-07T15:15:41Z",
    "updated_at": "2016-01-18T18:35:00Z",
    "pushed_at": "2017-08-21T18:23:26Z",
    "git_url": "git://github.com/SampleOrg/customrepo.git",
    "ssh_url": "git@github.com:SampleOrg/customrepo.git",
    "clone_url": "https://github.com/SampleOrg/customrepo.git",
    "svn_url": "https://github.com/SampleOrg/customrepo",
    "homepage": null,
    "size": 4682,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": "PHP",
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "open_issues_count": 0,
    "forks": 0,
    "open_issues": 0,
    "watchers": 0,
    "default_branch": "master",
    "permissions": {
      "admin": false,
      "push": true,
      "pull": true
    }
  },
...

How can I do that in a shell script with maybe an array or a for loop?

All I have tried is the following piece of code which reads the text file line by line and clones the repositories, but for creating this text file I need to do some processing priorly for filtering for clone_url param-values, but I want to skip this part entirely if possible.

while read p; do
    #echo $p
    git clone $p
done < Orgname_Repos.txt
0
8

Here you go:

#!/bin/bash
for i in $(curl "https://api.github.com/orgs/[organization]/repos?access_token=[access_token]" | grep -oP '"clone_url":\s*"\K[^"]+'); do
  echo git clone "$i"
done

This will output a list of the git clone commands to run, if you're happy just remove the echo to go for it.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.