mirror of
https://github.com/feder-cr/Jobs_Applier_AI_Agent.git
synced 2026-11-07 02:35:57 -10:00
Data sources: what each source's own licence says, and the credit they ask for
This commit is contained in:
feder-cr committed
1 parent
cc598fddd7
commit
a2dc29b491
3 files changed
+20
-14
No files matched your search
@@ -12,4 +12,6 @@ The published job database
|
||||
Is merged from the public sources named in pipeline/build.py, each with its home and licence as far as
|
||||
it states one; each release's manifest.json lists them, with the rows each gave or the error that left
|
||||
it out that day, and every row names the sources that carried it. Each posting belongs to its employer and links back to the original.
|
||||
[docs/data-sources.md](docs/data-sources.md) lists each source with its licence and how to ask for its removal.
|
||||
docs/data-sources.md lists each source with its licence and how to ask for its removal. AIHawk's MIT licence
|
||||
covers its code, not this data. Job data from Headcount Labs (https://headcountlabs.com), ODbL 1.0, with location
|
||||
data from OpenStreetMap contributors (ODbL) and Overture Maps. Forward Deployed Engineer jobs from FDE Times, CC BY 4.0.
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
AIHawk's job database merges 24 public sources. `pipeline/build.py` reads each one every day, and the release keeps the postings that survive the merge. Each posting names the sources that carried it.
|
||||
|
||||
The licence column says what the source's own page or dataset card states. "Not stated" means no licence was found on that page or in the repository's files. Nothing here is a legal opinion. Postings belong to their employers, and the link in each row goes to the page this table relies on.
|
||||
The licence column says what the source's own page or dataset card states. "Not stated" means no licence was found on that page or in the repository's files. Nothing here is a legal opinion. Postings belong to their employers, and the link in each row goes to the page this table relies on. AIHawk's MIT licence covers its code, not this data.
|
||||
|
||||
The repository does not state how many postings each source gives. The total in [README.md](../README.md) is for the whole database.
|
||||
|
||||
@@ -18,7 +18,7 @@ The repository does not state how many postings each source gives. The total in
|
||||
| `headcount-internships` | Internship postings | Hugging Face `HeadcountLabs/internships` | ODbL 1.0 | [dataset card](https://huggingface.co/datasets/HeadcountLabs/internships) |
|
||||
| `headcount-healthcare` | Healthcare job postings | Hugging Face `HeadcountLabs/healthcare-jobs` | ODbL 1.0 | [dataset card](https://huggingface.co/datasets/HeadcountLabs/healthcare-jobs) |
|
||||
| `eu-tech-jobs` | EU tech job postings, latest snapshot | Hugging Face `Aramente/eu-tech-jobs` | CC-BY-4.0 | [dataset card](https://huggingface.co/datasets/Aramente/eu-tech-jobs) |
|
||||
| `minmaxxxer` | Job postings with salaries | Hugging Face `minmaxxxer/job-postings-with-salaries` | Not stated. The card says "research-use" and gives no terms. It also says job descriptions belong to their employers. | [dataset card](https://huggingface.co/datasets/minmaxxxer/job-postings-with-salaries) |
|
||||
| `minmaxxxer` | Job postings with salaries | Hugging Face `minmaxxxer/job-postings-with-salaries` | Its own `LICENSE`: research, analysis, teaching and model development; "Do not republish the postings as job listings of your own." | [LICENSE](https://huggingface.co/datasets/minmaxxxer/job-postings-with-salaries/blob/main/LICENSE) |
|
||||
| `lookin4jobs` | Open job data in one JSON file | `lookin4jobs/open-jobs-data`, `data/jobs.json` | MIT (the repository's licence). The README says postings belong to their employers. | [open-jobs-data](https://github.com/lookin4jobs/open-jobs-data) |
|
||||
| `conorscode` | Open job data in one JSON file | `ConorsCode/open-jobs-data`, `data/jobs.json` | MIT (the repository's licence). The README says postings belong to their employers. | [open-jobs-data](https://github.com/ConorsCode/open-jobs-data) |
|
||||
| `fecklesspleb` | New jobs from ATS boards, in CSV | `fecklesspleb/ats-new-jobs-feed`, `data/jobs.csv` | MIT | [ats-new-jobs-feed](https://github.com/fecklesspleb/ats-new-jobs-feed) |
|
||||
@@ -27,12 +27,16 @@ The repository does not state how many postings each source gives. The total in
|
||||
| `vanshb03-new-grad` | New-grad job listings for 2027 | `vanshb03/New-Grad-2027`, `.github/scripts/listings.json` | MIT (`LICENSE.txt`) | [New-Grad-2027](https://github.com/vanshb03/New-Grad-2027) |
|
||||
| `vanshb03-internships` | Summer 2026 internship listings | `vanshb03/Summer2026-Internships`, `.github/scripts/listings.json` | MIT (`LICENSE.txt`) | [Summer2026-Internships](https://github.com/vanshb03/Summer2026-Internships) |
|
||||
| `simplify-trabaho` | Job listings in `data/listings.json` | `yanicells/SimplifyTrabaho` | MIT for the code. The README says the dataset is CC0 1.0 (public domain). | [SimplifyTrabaho](https://github.com/yanicells/SimplifyTrabaho) |
|
||||
| `headcount-engineering-interns` | Engineering internship postings, in CSV | `HeadcountLabs/engineering-internships`, `data/jobs.csv` | ODbL 1.0 for the job data (the repository's `LICENSE`). Job descriptions belong to their employers. | [engineering-internships](https://github.com/HeadcountLabs/engineering-internships) |
|
||||
| `headcount-software-interns` | Software engineering internship postings, in CSV | `HeadcountLabs/software-engineering-internships`, `data/jobs.csv` | ODbL 1.0 for the list data. Headcount Labs' own values are under DbCL 1.0. Job descriptions belong to their employers. | [software-engineering-internships](https://github.com/HeadcountLabs/software-engineering-internships) |
|
||||
| `headcount-engineering-interns` | Engineering internship postings, in CSV | `HeadcountLabs/engineering-internships`, `data/jobs.csv` | ODbL 1.0 for the job data (the repository's `LICENSE`). Job descriptions belong to their employers. Takedowns: [headcountlabs.com/remove](https://headcountlabs.com/remove/). | [engineering-internships](https://github.com/HeadcountLabs/engineering-internships) |
|
||||
| `headcount-software-interns` | Software engineering internship postings, in CSV | `HeadcountLabs/software-engineering-internships`, `data/jobs.csv` | ODbL 1.0 for the list data. Headcount Labs' own values are under DbCL 1.0. Job descriptions belong to their employers. Takedowns: [headcountlabs.com/remove](https://headcountlabs.com/remove/). | [software-engineering-internships](https://github.com/HeadcountLabs/software-engineering-internships) |
|
||||
| `applyguy` | 2027 new-grad jobs | `ApplyGuy/2027-New-Grad-Jobs`, `data/new-grad-jobs.json` | Not stated. No LICENSE file was found. | [2027-New-Grad-Jobs](https://github.com/ApplyGuy/2027-New-Grad-Jobs) |
|
||||
| `fde-jobs` | Forward-deployed engineer job list, in CSV | `stephanelx88/forward-deployed-engineer-jobs`, `data/fde-jobs.csv` | CC BY 4.0 | [forward-deployed-engineer-jobs](https://github.com/stephanelx88/forward-deployed-engineer-jobs) |
|
||||
| `remote-india` | Remote jobs open to India, in one JSON file | `raghavendra81422-boop/remote-jobs-open-to-india`, `data.json` | MIT (the repository's `LICENSE` file) | [LICENSE](https://raw.githubusercontent.com/raghavendra81422-boop/remote-jobs-open-to-india/HEAD/LICENSE) |
|
||||
|
||||
## Credit
|
||||
|
||||
Credit the sources ask for: job data from [Headcount Labs](https://headcountlabs.com), ODbL 1.0, with location data from OpenStreetMap contributors (ODbL) and Overture Maps; Forward Deployed Engineer jobs from FDE Times, CC BY 4.0.
|
||||
|
||||
## Removing a source
|
||||
|
||||
A source's owner can ask for it to be removed. There are two ways.
|
||||
|
||||
+9
-9
@@ -207,7 +207,7 @@ SOURCES = (
|
||||
Source("headcount-internships", "https://huggingface.co/datasets/HeadcountLabs/internships", "ODbL", _headcount("internships")),
|
||||
Source("headcount-healthcare", "https://huggingface.co/datasets/HeadcountLabs/healthcare-jobs", "ODbL", _headcount("healthcare-jobs")),
|
||||
Source("eu-tech-jobs", "https://huggingface.co/datasets/Aramente/eu-tech-jobs", "CC-BY-4.0", _eu_tech),
|
||||
Source("minmaxxxer", "https://huggingface.co/datasets/minmaxxxer/job-postings-with-salaries", "research use", lambda _: select(
|
||||
Source("minmaxxxer", "https://huggingface.co/datasets/minmaxxxer/job-postings-with-salaries", "research use, no republishing as listings", lambda _: select(
|
||||
f"read_parquet('{HF}/minmaxxxer/job-postings-with-salaries/resolve/main/job_postings_with_salaries.parquet')",
|
||||
url="coalesce(apply_url, url)", ats="ats", country="country_code", remote="workplace_type = 'remote'",
|
||||
salary_min="salary_min", salary_max="salary_max", salary_currency="salary_currency",
|
||||
@@ -219,32 +219,32 @@ SOURCES = (
|
||||
f"read_csv('{RAW}/fecklesspleb/ats-new-jobs-feed/HEAD/data/jobs.csv', header=true, all_varchar=true)",
|
||||
url="url", ats="ats", remote="workplace = 'remote'", posted_at="posted_at",
|
||||
)),
|
||||
Source("simplify-new-grad", "https://github.com/SimplifyJobs/New-Grad-Positions", "not found", _simplify("SimplifyJobs/New-Grad-Positions")),
|
||||
Source("simplify-internships", "https://github.com/SimplifyJobs/Summer2027-Internships", "not found", _simplify("SimplifyJobs/Summer2027-Internships")),
|
||||
Source("simplify-new-grad", "https://github.com/SimplifyJobs/New-Grad-Positions", "not stated", _simplify("SimplifyJobs/New-Grad-Positions")),
|
||||
Source("simplify-internships", "https://github.com/SimplifyJobs/Summer2027-Internships", "not stated", _simplify("SimplifyJobs/Summer2027-Internships")),
|
||||
Source("vanshb03-new-grad", "https://github.com/vanshb03/New-Grad-2027", "MIT", _simplify("vanshb03/New-Grad-2027")),
|
||||
Source("vanshb03-internships", "https://github.com/vanshb03/Summer2026-Internships", "MIT", _simplify("vanshb03/Summer2026-Internships")),
|
||||
Source("simplify-trabaho", "https://github.com/yanicells/SimplifyTrabaho", "MIT (code)", lambda _: select(
|
||||
Source("simplify-trabaho", "https://github.com/yanicells/SimplifyTrabaho", "CC0-1.0 (data), MIT (code)", lambda _: select(
|
||||
_unnested(f"{RAW}/yanicells/SimplifyTrabaho/HEAD/data/listings.json", "listings"),
|
||||
url="u.url", title="u.title", company="u.company", ats="u.source", location="array_to_string(u.locations, '; ')",
|
||||
remote="u.workSetup = 'remote'",
|
||||
)),
|
||||
Source("headcount-engineering-interns", "https://github.com/HeadcountLabs/engineering-internships", "not stated", lambda _: select(
|
||||
Source("headcount-engineering-interns", "https://github.com/HeadcountLabs/engineering-internships", "ODbL", lambda _: select(
|
||||
f"read_csv('{RAW}/HeadcountLabs/engineering-internships/HEAD/data/jobs.csv', header=true, all_varchar=true)",
|
||||
url="apply_url", location="locations", remote="remote = 'true'", posted_at="posted",
|
||||
)),
|
||||
Source("headcount-software-interns", "https://github.com/HeadcountLabs/software-engineering-internships", "not stated", lambda _: select(
|
||||
Source("headcount-software-interns", "https://github.com/HeadcountLabs/software-engineering-internships", "ODbL", lambda _: select(
|
||||
f"read_csv('{RAW}/HeadcountLabs/software-engineering-internships/HEAD/data/jobs.csv', header=true, all_varchar=true)",
|
||||
url="apply_url", location="locations", remote="remote = 'true'", posted_at="posted",
|
||||
)),
|
||||
Source("applyguy", "https://github.com/ApplyGuy/2027-New-Grad-Jobs", "not found", lambda _: select(
|
||||
Source("applyguy", "https://github.com/ApplyGuy/2027-New-Grad-Jobs", "not stated", lambda _: select(
|
||||
_unnested(f"{RAW}/ApplyGuy/2027-New-Grad-Jobs/HEAD/data/new-grad-jobs.json", "jobs"),
|
||||
url="u.url", title="u.title", company="u.company", location="u.location", posted_at="u.posted",
|
||||
)),
|
||||
Source("fde-jobs", "https://github.com/stephanelx88/forward-deployed-engineer-jobs", "not found", lambda _: select(
|
||||
Source("fde-jobs", "https://github.com/stephanelx88/forward-deployed-engineer-jobs", "CC-BY-4.0", lambda _: select(
|
||||
f"read_csv('{RAW}/stephanelx88/forward-deployed-engineer-jobs/HEAD/data/fde-jobs.csv', header=true, all_varchar=true)",
|
||||
url="fdetimes_url", country="country", salary_min="pay_low_usd", salary_max="pay_high_usd", posted_at="first_seen",
|
||||
)),
|
||||
Source("remote-india", "https://github.com/raghavendra81422-boop/remote-jobs-open-to-india", "not stated", lambda _: select(
|
||||
Source("remote-india", "https://github.com/raghavendra81422-boop/remote-jobs-open-to-india", "MIT", lambda _: select(
|
||||
_unnested(f"{RAW}/raghavendra81422-boop/remote-jobs-open-to-india/HEAD/data.json", "jobs"),
|
||||
url="u.roliva_url", title="u.title", company="u.company", location="u.location", remote="TRUE", posted_at="u.posted_date",
|
||||
)),
|
||||
|
||||
Reference in new issue
Block a user