Evaluation of the Regional and Remote Service Stations project 2016-2017
Overview
Service stations operate within a complex regulatory environment, in particular around the handling of hazardous chemicals. Service stations located in regional and remote areas face additional challenges in meeting regulatory requirements due to less frequent regulatory assistance and lack of awareness and understanding of regulatory requirements. In response to this, SafeWork NSW conducted a project targeting regional and remote service stations, with a particular focus on small independently owned service stations, to build their capability to self-comply. The project also included a Fair Trading component as part of a collaborative approach to cover regulatory requirements across multiple regulators, in this case in relation to Fair Trading price signage and FuelCheck registration requirements.
As part of this project, SafeWork inspectors visited 201 service stations in regional and remote New South Wales between December 2016 and January 2017 – this represents 20 per cent of service stations in those areas.
A comprehensive evaluation of the project has been conducted to assess the effectiveness of the project in terms of improving compliance and the effectiveness of the collaborative component.
Benefits of the evaluation
The evaluation findings were intended to identify opportunities to turn the project into Business As Usual, inform the design of collaborative projects, and generate insights for the regulators into the service station industry in regional and remote areas.
Evaluation methods
The evaluation relied on a mix of qualitative and quantitative methods. In-depth interviews were the core method to explore participants’ experience of the project. A total of 40 individuals were interviewed as part of the evaluation data collection phase: 16 businesses, 13 SafeWork NSW inspectors, 10 internal project stakeholders and a representative of the service industry peak body ACAPMA (Australian Convenience and Petroleum Marketers Association). Quantitative analysis of project data – in particular from Workplace Services Management System (WSMS), the SafeWork case management system – provided critical evidence about compliance issues encountered and the direct impact of the project on businesses capability in this regard.
The evaluation was conducted internally by evaluation staff not involved in the project, and the evaluation report has been externally peer-reviewed in particular by the NSW Government Centre for Program Evaluation.
Evaluation findings
- The inspectors’ audit tool, which was also sent to businesses ahead of the visits, was widely recognised as a good practice. Inspectors welcomed the visuals and level of details included in the audit tool, which has been used as an example by other jurisdictions (Qld, Tas) and as far as Nigeria.
- Businesses visited were an appropriate target with regard to their compliance needs and limited prior contact with regulators.
- Service stations in regional and remote New South Wales have a low level of compliance with regard to the manifest and the emergency plan; however most service stations visited were well equipped and trained around fire-fighting equipment and first aid.
- The project directly improved compliance through notices issued to one third of businesses visited. Out of the 66 businesses issued with a notice, 54 had fully complied (40) or partially complied (14) with all notices as at 17 July 2017. Most (60%) of the improvement notices issued by inspectors were in relation to hazardous chemicals.
- Most of the inspectors interviewed felt that the service stations visited had improved their level of compliance as a result of the visit. They were able to observe improvements made through follow-up visits to businesses. As part of the project, 78 follow-up visits were conducted, representing 39 per cent of businesses visited.
- The collaborative component of the project was limited to a Fair Trading checklist about price signage and FuelCheck registration. The information collected through this checklist showed that the majority of businesses were compliant with regard to price signage requirements and FuelCheck. Only 32 out of 192 businesses (16%) where the Fair Trading checklist was collected were referred for further inquiries.
- The main benefit from the collaborative component was around the efficient use of resources across regulators, with Fair Trading benefitting from “another set of eyes” through SafeWork inspectors. However, there is no clear evidence of benefits from the collaborative component for businesses: most of them confirmed that they didn’t have much contact in the past, which couldn’t lead to a reduction in touch points as initially intended.