data.gov.il runs on CKAN, so it has a real API rather than only file downloads. How to find a dataset's resource id, query it with datastore_search, and design around the two things that will break your job.
Key takeaways
- It is CKAN, which means the standard action API applies - the same endpoints and parameters documented for CKAN generally work here.
- The resource id, not the dataset name, is what you query against. Find it once, store it in configuration, and never hard-code it in the middle of your code.
- Paginate with limit and offset. A dataset that returns comfortably today will exceed your assumptions the moment the publisher adds a year of records.
- Government data changes shape without notice. Validate columns on every run and fail loudly rather than silently importing an empty or shifted dataset.
data.gov.il is the Israeli government's open data portal, and it runs on CKAN - an open-source platform with a full, documented API, so the data.gov.il API is CKAN's action API. Practically: there is no need to download files by hand or scrape the site. There are real endpoints to query, and that is the correct route.
Why it matters that this is CKAN
Many people visit the portal, download a CSV, and build a manual process around it. That works until the file is updated.
Because CKAN is a standard platform, its general documentation applies - meaning you are not dependent on bespoke documentation. The same actions documented for CKAN work here, accessible through the action API in the form /api/3/action/{action}.
Three actions cover almost every use:
| Action | What for |
|---|---|
package_search | Finding datasets by search terms |
package_show | Getting a dataset's details, including its resource list and their ids |
datastore_search | Fetching the rows themselves from a resource |
The key: resource id, not name
This is the first point of confusion, and the hierarchy is worth understanding:
- A dataset (package) - the item you see in the portal. It can contain several files.
- A resource - a single file within it. Every resource has a unique id.
And datastore_search works against a resource id, not a dataset name. The main parameters:
resource_id- requiredlimitandoffset- paginationq- free-text searchfields- which columns to returnsort- ordering
How to find the id: search for the dataset with package_search, then package_show to see its resources and their ids.
Then store it in configuration. Not in the middle of your code. A dataset republished as a new resource - which happens - gets a new id, and you want that to be a settings change rather than a deploy.
What are the data.gov.il API URLs and parameters?
Every data.gov.il API call is a GET to https://data.gov.il/api/3/action/{action} with query parameters, and every answer is JSON with success and result. Reading needs no API key. The calls a fetch job uses, with what each returned when tested on 2026-11-04:
| Call | Example | Returns |
|---|---|---|
package_list | /api/3/action/package_list | Every dataset name (1,206 on the test date) |
organization_list | /api/3/action/organization_list | The publishing bodies (61) |
package_search | ?q=<words>&rows=10 | count plus datasets, each with its resources, their ids and datastore_active |
package_show | ?id=<dataset name> | One dataset: licence, publisher, resources with download URL and last_modified |
datastore_search | ?resource_id=<id>&limit=1000&offset=0 | records, fields with their types, total, _links.next |
Only a resource with datastore_active: true answers datastore_search; the others are files you download. Useful parameters beyond paging:
filters- a JSON object of exact matches, for example{"shnat_yitzur": 2026}. Several fields are ANDed; a list of values matches any of them.fields- a comma-separated list of columns, which cuts the response size sharply on wide tables.sort- for exampleshnat_yitzur desc.limit=0- returns onlytotaland the field list, a cheap row count before a full pull.
Hebrew values in q and filters must be URL-encoded.
What limits does the data.gov.il API have?
The data.gov.il datastore does not behave like CKAN's defaults, so measure before you design. What a test against the vehicle licensing resource showed on 2026-11-04:
- Default page size. CKAN documents a default
limitof 100 and a maximum of 32,000 unless the site configures otherwise. A request with nolimithere returned 32,000 rows, andlimit=100000was accepted and returned 100,000 rows (about 57 MB, in under 8 seconds). Always send an explicitlimit. - Estimated totals. On the unfiltered 4.2 million row table,
total_was_estimatedwastrue; with a filter the total was exact. Do not use an unfilteredtotalas a completeness check. - Errors are HTML, not JSON. A wrong
resource_id, an unknown column infieldsand a missing dataset all returned HTTP 404 with an HTML page instead of CKAN's JSON error. Check the status code and content type before parsing. - SQL is not a route. A
datastore_search_sqlquery against a resource returned the same HTML 404, so plan the job ondatastore_search.
The two things that will break your job
This is what separates a script that ran once from a process you can rely on.
1. The structure changes without notice
Government data is published by different bodies, and there is no guarantee of structural stability. A column can be renamed, disappear, or be added - between runs. And nobody will tell you.
What actually happens: the script runs, does not crash, and imports zero rows or rows with empty fields. That silent failure is worse than a crash, because it surfaces weeks later in a report that looks odd.
The defence: validate the columns at the start of every run. If a column the process depends on is missing - stop and alert, do not continue. And alert too when the row count drops suddenly; going from 40,000 to 12 is not an update, it is a problem.
2. Data quality is not uniform
Different sources, different standards. In Israeli data that shows up in a few recurring patterns:
- Dates in mixed formats within the same column.
- Numeric fields as text - with commas, spaces or a currency symbol.
- Hebrew with double spaces and invisible characters at the edges. Two values that look identical and do not compare as strings.
- Missing values represented several ways - empty, a dash, "unknown".
The conclusion: a normalisation layer is not a luxury. Clean and convert types at ingestion, before the data touches the rest of the system.
What you are allowed to do with it
The data on the portal was published for public use, and that is what distinguishes this from scraping a website: there is an official API intended for exactly this.
Two practical caveats:
- Each dataset carries its own licence. It appears in the dataset details and is worth reading - particularly if you are building a commercial product on top of the data.
- Fetch at a reasonable rate. This is public infrastructure. Paginated requests with pauses beat a hundred parallel ones, and it is also what keeps your own process stable.
The pattern that works
- Discover once: find the dataset and its resource ids, and store them in configuration.
- Fetch in pages with
limitandoffset, not everything at once. - Validate the structure at the start of every run - and stop if it changed.
- Normalise at ingestion - dates, numbers, Hebrew text.
- Keep a raw copy of what arrived. When something looks odd in a month, that is the evidence of what actually came through.
- Alert on a sharp change in row count.
And store the data on your side rather than fetching live on every request. A public portal is not your database, and its availability is not your responsibility - but your process is.
Why this is useful
The portal holds thousands of public datasets. The common uses I have seen:
- Data enrichment - cross-referencing internal records against a public source.
- Market research - statistical data by region or sector.
- Monitoring - watching a dataset that updates and alerting on change.
And in each of them, the expensive part is not the fetching - it is normalisation and reliability. The fetch itself you will finish in an hour.
Sources
Frequently asked questions
Does data.gov.il have an API?
Yes. The portal runs on CKAN, so the standard CKAN action API applies through paths of the form /api/3/action/{action}. The three actions covering most uses are package_search to find datasets, package_show to see a dataset's resources and their ids, and datastore_search to fetch the rows themselves.
What is a resource id and where do I find it?
A dataset can contain several files, and each file is a resource with its own unique id. datastore_search queries against that resource id rather than the dataset name. Find it by searching with package_search and then calling package_show to list the dataset's resources - then store the id in configuration rather than hard-coding it, since a republished dataset gets a new one.
Why does my government data import silently return nothing?
Almost always because the dataset's structure changed - a column renamed, removed or added - without any notice. The script does not crash; it imports zero rows or rows with empty fields, and that surfaces weeks later in a report that looks wrong. Validate expected columns at the start of every run and stop loudly, and alert when the row count drops sharply.
Is using data.gov.il the same as scraping?
No - there is an official API built for programmatic access, and the data was published for public use. Two caveats remain: each dataset carries its own licence, which appears in the dataset details and is worth reading if you are building a commercial product on it, and you should fetch at a reasonable rate because this is shared public infrastructure.
What data quality problems should I expect from Israeli open data?
Mixed date formats within one column, numeric fields stored as text with commas or currency symbols, Hebrew values carrying double spaces and invisible characters so that two apparently identical values do not compare equal, and missing values represented several different ways. A normalisation layer at ingestion is not optional - clean and convert types before the data reaches the rest of your system.
Do I need an API key for the data.gov.il API?
No key is needed to read. package_search, package_show and datastore_search on public datasets answer plain GET requests to https://data.gov.il/api/3/action/. Keep the request rate reasonable, because it is public infrastructure, and store the resource ids you use in configuration.
How many rows does datastore_search return per request on data.gov.il?
In a test on 2026-11-04, a request with no limit returned 32,000 rows, while CKAN's documented default is 100. A limit of 100,000 was accepted and returned about 57 MB. Send an explicit limit, such as 1,000 to 10,000, and page with offset or _links.next.
Why does data.gov.il return an HTML page instead of JSON?
In testing, every failed call - a wrong resource_id, an unknown column in fields, a dataset that does not exist, a datastore_search_sql query - returned HTTP 404 with an HTML error page rather than CKAN's JSON error object. Check the status code and the content type before parsing, and log the request URL.
Keep reading
Related service
Integrations
Make the systems you already pay for talk to each other.
Have a project like this?
Tell me what you're trying to automate or build. I reply within 24 business hours with a few targeted questions, then we walk through it on a free 30-minute call, with no commitment. You come away with a scope, a timeline and a fixed price - or a straight answer that it isn't worth building.