Collated air quality data
Dear Environmental Protection Department,
I hope to investigate air quality in Hong Kong over time and I am
pleased to see that you provide such data - however, the format
that is provided is somewhat arduous to collate.
While you provide monthly collations of the hourly AQHI for all
monitoring stations via the Data.Gov.HK website - and I commend you
on your adoption of that portal - for the full breakdown of the
data that comprises the AQHI e.g. fine and respirable suspended
particulate values, one apparently must manually download the data
by selecting each of 15 stations, in monthly increments. If one is
to collate all the data, which goes back to 1999, that is roughly
15x12x16 years or 2880 forms to fill on your website.
If you have an API for your data service, then this becomes a much
more feasible proposition but I have not seen indication of one -
and the Java Server system you use for your website is not easy to
'scrape' i.e. to replicate the function of an API where there is
none.
Therefore, I request that you make available to me, data for all
monitoring stations, hourly, for all pollutant measures for all
dates. I also humbly suggest that you consider making the data more
readily available in this large scale via your web services.
I thank you for your time taken in this matter.
Yours faithfully,
Rob Davidson
I will be out of the office starting 02/09/2015 and will not return until
07/09/2015.
For urgent matter, please contact Ms Maggie Ho / E(KM)1 at tel: 2594 6083.
Thanks.
This notice is generated in response to your mail:
From: Rob Davidson <[FOI #35 email]>
Subject: Freedom of Information request - Collated air quality data
Date: 9/6/2015 12:13:29 PM
Dear Mr Davidson,
Thank you for your email dated 6/9/2015 seeking air monitoring records.
The records you requested are available for free on our Environmental
Protection Interactive Centre (EPIC) website at
<[1]http://www.epd.gov.hk/epd/epic/english/d...> for public
download. Please note that when downloading data from the EPIC website,
you may choose the "hourly" averaging period which allows you to download
up to 366 days per monitoring station at one time. The limits on different
averaging time ranges are provided for your reference below.
* Hour: no more than 366 days
* Day: no more than 31 days
* Month: no more than 60 months
* Year: no more than 5 years
The long term trends of air pollutants are also available in our annual
air quality reports.
URL:
[2]http://www.aqhi.gov.hk/en/download/air-q...
We thank your valuable suggestion of making large-scale data sets more
readily available on our portal. We'll take a serious consideration on it
and constantly review our services to improve our web services on data
access.
If you would like us to extract and collate the data in the format per
your request, charges will be incurred to cover additional staff efforts.
The charges will be based on the estimated time of staff efforts and the
hourly rate of staff. If you would like to proceed with this request,
please confirm by replying to this email. The Department will then
provide you with the estimated charge for the provision of all hourly air
pollutant data at our monitoring stations as available on our EPIC
website. If you accept the charge, we will issue a demand note and the
data extraction will start once the full payment is received. If you would
not like to proceed with this application, please reply to confirm the
withdrawal of the request.
Should you have any questions, please do not hesitate to contact us again.
Yours sincerely,
Jacquie Hui
for Director of Environmental Protection
Dear Environmental Protection Department,
Thank you for the new URL and suggestion that I download the data in stages from your website.
Using the download options via this new URL, I appear to have two scenarios:
The first is downloading 6 stations of data at a time, but only for 1 parameter (e.g. carbon monoxide or nitrogen dioxide) and for 1 year at a time. With 15 stations, 7 parameters and 20 years of data, I would have to fill out your form 3*7*20 times, or 420 times.
The alternative is downloading 1 station at a time but for all parameters, 1 year at a time. That makes a repetitive task of 15 stations *1 (all parameters together) *20 years or 300 times.
I hope you can understand my reluctance to spend several hours on your website, downloading data that I assume someone in your office could access with one basic SQL request to the underlying database in a matter of moments.
I therefore request a breakdown of the charges proposed by you for this task.
Yours faithfully,
Rob Davidson
Dear Mr Davidson,
The estimated cost to provide you with all hourly pollutant concentrations
at all monitoring stations as available on EPIC is $2,500. A breakdown of
charges is provided below for your reference.
The data files will be provided to you via e-mail in comma-delimited
format. If you accept the charges, the Department will issue a demand
note. The data extraction will start once the full payment is received.
If you would not like to proceed with this application, please reply to
confirm the withdrawal of the application.
Please feel free to contact us if you have any questions.
Sincerely,
Jacquie Hui
for Director of Environmental Protection
Rob Davidson To [email address]
<[FOI #35 email]> cc
Subject Re: Collated air quality
15/09/2015 16:21 data [Ref. EPD449/33/2015]
Dear Environmental Protection Department,
Thank you for the new URL and suggestion that I download the data in
stages from your website.
Using the download options via this new URL, I appear to have two
scenarios:
The first is downloading 6 stations of data at a time, but only for 1
parameter (e.g. carbon monoxide or nitrogen dioxide) and for 1 year at a
time. With 15 stations, 7 parameters and 20 years of data, I would have to
fill out your form 3*7*20 times, or 420 times.
The alternative is downloading 1 station at a time but for all parameters,
1 year at a time. That makes a repetitive task of 15 stations *1 (all
parameters together) *20 years or 300 times.
I hope you can understand my reluctance to spend several hours on your
website, downloading data that I assume someone in your office could
access with one basic SQL request to the underlying database in a matter
of moments.
I therefore request a breakdown of the charges proposed by you for this
task.
Yours faithfully,
Rob Davidson
Dear Environmental Protection Department,
Thank you for providing details of the amount you would want to charge for providing air quality data under the Code on Access to Information.
I must admit to being quite shocked by the charges. I also feel that the breakdown of costs - charging an hourly rate for staff time, for a task that will take a couple of hours - is somewhat outwith the spirit of the Code.
In the Code, under the description of charges that might be made, the example is given of photocopying expenses. This may be somewhat dated now but I believe it indicates that the cost of reproduction may be charged - where consumable such as paper or more modernly, CD/DVD's or USB hard drives might be required for conveying the information from the stored copy to the requester. As we are discussing the transfer of electronic files by email, there is no such cost involved for your department on this occasion.
The more general wording of the Code does use the term 'resources' which could be applied to the human resources for which you want to charge, but I feel that a charge for staff time ought only to be applied in exceptionally arduous and time consuming cases. On this occasion, I seriously suspect you have overestimated the time requirements for this task that ought to take 15 minutes, but even if it were to take 4 hours to query one table in a database, I do not believe that would be chargeable under the spirit of the code. If it was the case then the code would not only provide details about the cost per page for photocopying, but also the expected cost for salary for not only photocopying the pages but finding the files and licking the stamps prior to posting the envelope.
I hope you can take the ending of the last paragraph in jest, but I also urge you to reconsider the charges with due concern for the spirit of the Code which indicates that public bodies ought to be willing to take some steps to provide the data to the public for free.
Yours faithfully,
Rob Davidson
Dear Mr Davidson,
Thank you for your email dated 17 September 2015 regarding the access on
air quality data.
As you are well aware, the air quality data you requested are readily
available for free on a public portal for your free access. According to
paragraph 1.14 of the Code on Access to Information (the Code), the Code
does not oblige departments to provide the requested information which is
already published.
For your specific case, we can make our best endeavors to accede your
request to retrieve the data to your specific format. However, unlike your
suggestion, the retrieval and collation cannot be done in a short time.
Similar to your situation for downloading the data from our EPIC centre,
it is time consuming for us to retrieve and download the data properly to
the format specified and ensure the data accuracy. All these would
result in additional resources.
The charge estimate is based on the actual work required and we would like
to seek your understanding that time and efforts are essential for
providing the data in this case as well as the format specified.
Consistent with the spirit of the Code, the online EPIC portal is
available to facilitate public access of air quality data and specific
request like in your case justify charging based on the actual work done
required in the circumstances.
Please feel free to contact us if you confirm your agreement to pay the
charges for reproduction of the requested data or have further enquiry.
Sincerely,
Jacquie Hui
for Director of Environmental Protection
Rob Davidson To [email address]
<[FOI #35 email]> cc
Subject Re: Collated air quality
17/09/2015 11:08 data [Ref. EPD449/33/2015]
Dear Environmental Protection Department,
Thank you for providing details of the amount you would want to charge for
providing air quality data under the Code on Access to Information.
I must admit to being quite shocked by the charges. I also feel that the
breakdown of costs - charging an hourly rate for staff time, for a task
that will take a couple of hours - is somewhat outwith the spirit of the
Code.
In the Code, under the description of charges that might be made, the
example is given of photocopying expenses. This may be somewhat dated now
but I believe it indicates that the cost of reproduction may be charged -
where consumable such as paper or more modernly, CD/DVD's or USB hard
drives might be required for conveying the information from the stored
copy to the requester. As we are discussing the transfer of electronic
files by email, there is no such cost involved for your department on this
occasion.
The more general wording of the Code does use the term 'resources' which
could be applied to the human resources for which you want to charge, but
I feel that a charge for staff time ought only to be applied in
exceptionally arduous and time consuming cases. On this occasion, I
seriously suspect you have overestimated the time requirements for this
task that ought to take 15 minutes, but even if it were to take 4 hours to
query one table in a database, I do not believe that would be chargeable
under the spirit of the code. If it was the case then the code would not
only provide details about the cost per page for photocopying, but also
the expected cost for salary for not only photocopying the pages but
finding the files and licking the stamps prior to posting the envelope.
I hope you can take the ending of the last paragraph in jest, but I also
urge you to reconsider the charges with due concern for the spirit of the
Code which indicates that public bodies ought to be willing to take some
steps to provide the data to the public for free.
Yours faithfully,
Rob Davidson
Dear Environmental Protection Department,
I am sad to see that you are unwilling to facilitate public use of your data. As you point out the data is already available online, but I'm afraid that we disagree on the concept of 'readily' available data.
As I've pointed out, I would need to fill out the online form 300 times in order to download all the data. You seem to think this is acceptable and a helpful way to publish data.
I still do not understand why it would take you 4 hours to extract the data from the underlying database or why even this level of activity would be chargeable under the code. I can only rationalise such a length of time if this, like your website, is due to inefficient practices within your office. Perhaps worse, I fear that it may indicate a lack of willingness to facilitate public use of the public sector information in your care.
With these concerns in mind, I will follow up this email with another, asking for an internal review of your decisions, again in line with the Code.
Yours faithfully,
Rob Davidson
Dear Environmental Protection Department,
Please pass this on to the person who conducts Freedom of Information reviews.
I am writing to request an internal review of Environmental Protection Department's handling of my FOI request 'Collated air quality data'.
The position of your department has been that I ought to pay an inordinate amount of money for what ought to be a simple, short task. When queried about the excessive cost, the answer has been to hide behind the fact that the data is available online already but, as I've hopefully made clear during my correspondence, the current system for accessing the data is obtuse and would require an inordinate amount of time to access all the data (300 submissions of a web form). I am concerned that the department is not acting in accordance with the spirit of the code - to facilitate public access to public sector information and is instead hiding behind the letter of the code and thwarting meaningful public access to such data.
A full history of my FOI request and all correspondence is available on the Internet at this address: https://accessinfo.hk/en/request/collate...
Yours faithfully,
Rob Davidson
Dear Mr Davidson,
Your request for internal review has been received. We will processing
your request and will come back to you later. Thank you.
Yours sincerely,
Louis Chan
Access to Information Officer
Environmental Protection Department
Dear Mr Davidson,
Thank you for your message. We will continue reviewing our services to
the public for further improvement.
Please feel free to contact us if you have other queries on air quality
data.
Sincerely,
Jacquie Hui
for Director of Environmental Protection
Rob Davidson To [email address]
<[FOI #35 email]> cc
Subject Re: Collated air quality
28/09/2015 08:10 data [Ref. EPD449/33/2015]
Dear Environmental Protection Department,
I am sad to see that you are unwilling to facilitate public use of your
data. As you point out the data is already available online, but I'm
afraid that we disagree on the concept of 'readily' available data.
As I've pointed out, I would need to fill out the online form 300 times in
order to download all the data. You seem to think this is acceptable and a
helpful way to publish data.
I still do not understand why it would take you 4 hours to extract the
data from the underlying database or why even this level of activity would
be chargeable under the code. I can only rationalise such a length of time
if this, like your website, is due to inefficient practices within your
office. Perhaps worse, I fear that it may indicate a lack of willingness
to facilitate public use of the public sector information in your care.
With these concerns in mind, I will follow up this email with another,
asking for an internal review of your decisions, again in line with the
Code.
Yours faithfully,
Rob Davidson
Dear Mr Davidson,
I refer to your email of 28 September 2015 requesting this department to
conduct an internal review for your earlier information request.
Our office is now handling your review request according to para. 1.25 of
the Code on Access to Information. We would try our best to notify you
the outcome of review within 21 days from the date of your email. Thank
you.
Yours sincerely,
Louis Chan
Access to Information Officer
Environmental Protection Department
Dear Mr Davidson,
I refer to your email of 28 September 2015 requesting this department to
conduct an internal review for your earlier information request.
Further to our interim reply dated 8 October 2015, I would like to inform
you that we need a bit more time to complete the review process. We
would try out best to complete the review and notify you the outcome as
soon as possible. Thank you.
Yours sincerely,
Louis Chan
Access to Information Officer
Environmental Protection Department
Dear Mr. Davidson,
I refer to your email dated 28 September 2015, requesting us to conduct an
internal review on the handling of your request for providing all hourly
air monitoring data in the past. After carefully reviewing your request,
we consider that your request for us to amalgamate all monitoring data in
one single database by preparing a script to extract data falls outside
the scope of the Code on Access to Information and would not be acceded
to.
According to Section 1.14 of the Code on Access to Information, "the Code
does not oblige departments to - provide on request information which is
already published, either free or at a charge." The Guidelines on
Interpretation and Application further elaborate in Section 1.14.3 that
"where the information requested is already available in published form an
applicant can simply be referred to the appropriate source.". Moreover,
paragraph 1.14.4 of the Guidelines further explains that "the requirement
to pay for certain information under a charged service is not to be
circumvented by way of a request for information under the provisions of
the Code".
As we advised in our last email of 15 September 2015, the data on air
quality that you requested is readily available for free at the
Environmental Protection Interactive Centre (EPIC) data portal. However
your specific request goes beyond access to data but involves compiling
publicly available data in a format that you require. In our last email
reply dated 16 September 2015, we once quoted a charge of $2,500 for
performing the task you sought. Upon review, we consider this task indeed
constitutes a private service. We consider it is not the best use of
public resources for providing a tailor-made service compiling publicly
available data for a private entity.
If you are not satisfied with the above decision, you may complain to The
Ombudsman, whose address is-
30/F, China Merchants Tower
Shun Tak Centre
168-200 Connaught Road Central
Hong Kong
Telephone : 2629 0555
Fax : 2882 8149
Sincerely,
Shermann Fong
Principal Environmental Protection Officer
for Director of Environmental Protection
(Tel: 2594 6300, Email: [電郵地址])
Louis SH CHAN/EPD/HKSARG To [索取資料要求 #35 電郵]
S[KM] cc
Subject Fw: Internal review of Freedom of
20/10/2015 11:09 Information request - Collated air
quality data [EPD Ref: EPD449/33/2015]
Dear Mr Davidson,
I refer to your email of 28 September 2015 requesting this department to
conduct an internal review for your earlier information request.
Further to our interim reply dated 8 October 2015, I would like to inform
you that we need a bit more time to complete the review process. We
would try out best to complete the review and notify you the outcome as
soon as possible. Thank you.
Yours sincerely,
Louis Chan
Access to Information Officer
Environmental Protection Department
Guy Freeman left an annotation ()
Charges can be made (see Section 1.24 of the Code at http://www.access.gov.hk/en/code.htm#cha...), but "Any charges levied will [should?] reflect the cost of providing the information".