 <?xml-stylesheet type="text/css" href="https://www.esdm.co.uk/Data/style/rss1.css" ?> <?xml-stylesheet type="text/xsl" href="https://www.esdm.co.uk/Data/style/rss1.xsl" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
  <channel>
    <title>The knowledge base blog</title>
    <link>https://www.esdm.co.uk/knowledge</link>
    <description />
    <docs>http://www.rssboard.org/rss-specification</docs>
    <generator>mojoPortal Blog Module</generator>
    <language>en-GB</language>
    <ttl>120</ttl>
    <atom:link href="https://www.esdm.co.uk/Blog/RSS.aspx?p=138~108~10" rel="self" type="application/rss+xml" />
    <itunes:owner />
    <itunes:explicit>no</itunes:explicit>
    <item>
      <title>Opening ESRI Personal Geodatabases in QGIS</title>
      <description><![CDATA[<p>In the more recent 64-bit versions of QGIS it is no&nbsp;longer possible&nbsp;to open ESRI personal geodatabases using the 'Add vector layer' functionality. However, it is possible to enable access to personal geodatabases by doing the following:</p>

<ol>
	<li>Download the 64-bit version on the MS Access ODBC driver (AccessDatabaseEngine_x64.exe): <a href="http://www.microsoft.com/en-gb/download/details.aspx?id=13255">http://www.microsoft.com/en-gb/download/details.aspx?id=13255</a></li>
	<li>If you <strong>don’t</strong> have 32-bit MS Office installed you can just run the exe, however if you <strong>do</strong> have 32-bit office installed then you need to start an admin command prompt and run the installer in passive mode as follows:</li>
</ol>

<pre class="Indent1">
AccessDatabaseEngine_x64.exe /passive</pre>

<ol start="3">
	<li>For QGIS 2: Then you need to edit the qgis-ltr.bat (normally in C:\Program Files\QGIS x.xx\bin, in previous versions used to be qgis.bat) and add the following lines (before the ‘start’ command)</li>
</ol>

<pre class="Indent1">
set OGR_SKIP=ODBC
set PGEO_DRIVER_TEMPLATE=DRIVER=Microsoft Access Driver (*.mdb, *.accdb);DBQ=%%s</pre>

<ol start="4">
	<li>For QGIS 3: In QGIS Settings panel -&gt; Options | System | Environment add the following two variables:
	<p>variable name: PGEO_DRIVER_TEMPLATE&nbsp;<br />
	value: DRIVER=Microsoft Access Driver (*.mdb, *.accdb);DBQ=%s<br />
	Or if being used on a shared server add a system environment variable&nbsp;<br />
	PGEO_DRIVER_TEMPLATE=DRIVER=Microsoft Access Driver (*.mdb, *.accdb);DBQ=%s</p>
	</li>
	<li>
	<div>Now you should be able to start QGIS and simply drag the geodatabase mdb file onto the QGIS workspace and it will open.</div>
	</li>
</ol>

<div>Note: This solution has been tested using QGIS v2.14.11 (Essen)&nbsp; &amp; QGIS v3.4.13 (Madeira)</div>

<div>Note: This post is based on information taken from the web page listed below, however that page&nbsp;was based on an older version of QGIS (2.8.1) which used the qgis.bat file rather than qgis-ltr.bat.</div>

<div><a href="http://gis.stackexchange.com/questions/129514/opening-esri-personal-geodatabase-mdb-using-qgis">http://gis.stackexchange.com/questions/129514/opening-esri-personal-geodatabase-mdb-using-qgis</a></div>
<br /><a href='https://www.esdm.co.uk/opening-esri-personal-geodatabases-in-qgis'>Steve Ellwood</a>&nbsp;&nbsp;<a href='https://www.esdm.co.uk/opening-esri-personal-geodatabases-in-qgis'>...</a>]]></description>
      <link>https://www.esdm.co.uk/opening-esri-personal-geodatabases-in-qgis</link>
      <author>stevee@esdm.co.uk (Steve Ellwood)</author>
      <comments>https://www.esdm.co.uk/opening-esri-personal-geodatabases-in-qgis</comments>
      <guid isPermaLink="true">https://www.esdm.co.uk/opening-esri-personal-geodatabases-in-qgis</guid>
      <pubDate>Wed, 01 Mar 2017 10:13:00 GMT</pubDate>
    </item>
    <item>
      <title>QGIS field calculator backslash gotcha</title>
      <description><![CDATA[<p>Quantum GIS has a powerful field calculator, similar to that in ArcGIS, and <a href="http://hub.qgis.org/wiki/quantum-gis/Calculating_field_values" target="_blank">described here</a>. I was using it to update a tile index shapefile for use with GeoServer and a folder full of georeferenced TIFF images. The shapefile comprises polygons each representing the extent of one of the tiled images, having a [location] attribute containing the path to the image.</p> <p>After moving things around, the path was no longer correct, so I needed to update (for example) “'D:\MapData\OrdnanceSurvey\streetview\HP40NE.TIF” to “N:\OrdnanceSurvey\streetview\HP40NE.TIF”, with a similar replacement occurring across a few thousand rows.</p> <p>So in the QGIS field calculator, I tried to set the field to this expression:</p> <p>replace("location" , 'D:\MapData\' , 'N:\')</p> <p><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="QGIS field calculator grumbling about syntax" border="0" alt="QGIS field calculator grumbling about syntax" src="https://www.esdm.co.uk/Data/Sites/1/media/wlw/image_35.png" width="508" height="480"></p> <p>But at the bottom of the screen we can see that QGIS is not happy with this expression, and clicking “more info” tells us </p> <p><strong>Parser Error:</strong><br>syntax error, unexpected COLUMN_REF, expecting COMMA or ')'</p> <p>The solution was to escape the backslash characters like this:</p> <p>replace("location" , 'D:\\MapData\\' , 'N:\\')</p> <p>I shall internalize my feelings about whether or not this should have been necessary, or the helpfulness of the error message and the documentation about this feature – hey QGIS is free!</p><br /><a href='https://www.esdm.co.uk/qgis-field-calculator-backslash-gotcha'>Crispin Flower</a>&nbsp;&nbsp;<a href='https://www.esdm.co.uk/qgis-field-calculator-backslash-gotcha'>...</a>]]></description>
      <link>https://www.esdm.co.uk/qgis-field-calculator-backslash-gotcha</link>
      <author>crispin.flower@idoxgroup.com (Crispin Flower)</author>
      <comments>https://www.esdm.co.uk/qgis-field-calculator-backslash-gotcha</comments>
      <guid isPermaLink="true">https://www.esdm.co.uk/qgis-field-calculator-backslash-gotcha</guid>
      <pubDate>Thu, 28 Feb 2013 15:38:40 GMT</pubDate>
    </item>
    <item>
      <title>Inspecting web service requests from Quantum GIS with Fiddler</title>
      <description><![CDATA[<p>When connecting to any mapping web service, such as a WMS of WFS, just occasionally things don’t work first time (or is that just me…?).&nbsp; When this happens it is useful to be able to inspect the HTTP requests/responses for clues.</p> <p>Our HTTP inspection tool of choice is of course <a href="http://www.fiddler2.com" target="_blank">Fiddler</a>, however if you open this while using QGIS to connect to a web service, you see… nothing. So you need to tell QGIS to pass requests through a proxy, using Host = localhost and Port 8888 as shown below:</p> <p>QGIS menu Settings &gt; Options &gt; Network:</p> <p><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="QGIS proxy options" border="0" alt="QGIS proxy options" src="https://www.esdm.co.uk/Data/Sites/1/media/wlw/image_7.png" width="644" height="416"></p> <p>Requests now start appearing in Fiddler.</p> <p>However, there’s a small gotcha… once set into this mode, requests fail entirely if Fiddler is not running.&nbsp; A WMS will simply fail to draw, and connecting to a WMS server gives this error:</p> <blockquote> <p>Could not understand the response. The wms provider said:  <p>Download of capabilities failed: Connection refused</p></blockquote> <p>This cost me half an hour this morning, having forgotten that I had set QGIS into this mode a few days ago, grrr.&nbsp; Here we are after I’d remembered the solution, showing a WMS of archaeological sites recorded by the National Trust, in Snowdonia, Wales.</p> <p><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="Archaeological sites in Snowdonia, Wales" border="0" alt="Archaeological sites in Snowdonia, Wales" src="https://www.esdm.co.uk/Data/Sites/1/media/wlw/image_9.png" width="496" height="484"></p><br /><a href='https://www.esdm.co.uk/inspecting-web-service-requests-from-quantum-gis-with-fiddler'>Crispin Flower</a>&nbsp;&nbsp;<a href='https://www.esdm.co.uk/inspecting-web-service-requests-from-quantum-gis-with-fiddler'>...</a>]]></description>
      <link>https://www.esdm.co.uk/inspecting-web-service-requests-from-quantum-gis-with-fiddler</link>
      <author>crispin.flower@idoxgroup.com (Crispin Flower)</author>
      <comments>https://www.esdm.co.uk/inspecting-web-service-requests-from-quantum-gis-with-fiddler</comments>
      <guid isPermaLink="true">https://www.esdm.co.uk/inspecting-web-service-requests-from-quantum-gis-with-fiddler</guid>
      <pubDate>Mon, 04 Jun 2012 07:33:04 GMT</pubDate>
    </item>
    <item>
      <title>Some jottings on SQL Azure and GIS</title>
      <description><![CDATA[<p>I have set up a SQL Azure 90 day trial (as of 18th November) to test whether ArcSquirrel can work with Azure. This was prompted by an email from Robert Cheetham, CEO at Azavea, who is investigating cloud solutions for their Cicero product. Here are some notes about</p>

<ol>
	<li>setting up and accessing SQL Azure databases</li>
	<li>using SQL Azure with ArcGIS + ArcSquirrel and MapLink</li>
	<li>using SQL Azure with Quantum GIS and MapServer.</li>
</ol>

<h4>Azure Management Portal</h4>

<p>Although I’ve never set up any cloud services before (other than <a href="http://www.arvixe.com/" target="_blank" title="Arvixe web site">Arvixe web hosting</a>) it was incredibly easy to setup a trial SQL Azure subscription (“Windows Azure Platform Introductory Special”), create a database, and add some firewall rules to control access. I didn’t dig deep, but the “Management Portal” seems nice and simple:</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_10.png"><img alt="Azure Management Portal" border="0" height="188" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_4.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="Azure Management Portal" width="244" />&nbsp;</a></p>

<p>It helpfully tells you your connection string for ADO.Net, ODBC and PHP (passwords not included!):</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_8.png"><img alt="SQL Azure connection strings" border="0" height="180" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_3.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="SQL Azure connection strings" width="244" />&nbsp;</a></p>

<p>See this page for <a href="http://www.microsoft.com/windowsazure/offers/popup/popup.aspx?lang=en&amp;locale=en-gb&amp;offer=ms-azr-0018p&amp;no-rewrite=true" target="_blank" title="Details of the Windows Azure Platform Introductory Special">details of the Windows Azure Platform Introductory Special</a> including what you get for no charge and the pricing for the rest.</p>

<h4>MapLink, ArcSquirrel and Azure</h4>

<p>Hartmut connected up our MapLink to the database and pushed some spatial data into my Azure database from ArcGIS – 20,000 polygons took about 75 minutes.&nbsp; Hartmut had the layers drawing in ArcMAP, performing as well as in a normal SQL Server instance apparently, with only minor mods to our code to make the initial connection. However editing will not work without some more tweaks - MapLink and ArcSquirrel rely on some SQL metadata that is a little different on Azure.</p>

<h4>SQL Server Management Studio and MS Access</h4>

<p>Connecting with SSMS is simple, however you get reduced functionality – for example you cannot right-click &gt; edit top 200 records or anything, so quick edits of a small table need to be done with SQL or in MS Access or some other client. Fortunately connecting with MS Access (ODBC + SQL Native Client 10) is also simple, and&nbsp; the Azure database behaves just like a local SQL instance – remarkable querying speed (all things considered), though slower inserts (10,000 records took something like 15 minutes). NB you cannot directly link a spatial table into Access because it doesn’t recognize the geometry data type; but to update attributes you can link a view.</p>

<p>Also, in SSMS you get no designer tools, e.g. for creating a view; everything has to be done with T-SQL.</p>

<h4>SQL Azure Database Manager</h4>

<p>After writing the above I found the database manager in the Azure control panel, which has functionality similar to SSMS, including the ability to edit the tables*:</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_12.png"><img alt="SQL Azure database manager" border="0" height="154" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_5.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="SQL Azure database manager" width="244" />&nbsp;</a></p>

<p>* spatial data types not supported for view or edit.</p>

<p>You get a designer for creating/modifying tables, but views and stored procedures are T-SQL only.</p>

<h4>MapServer, OGR and Azure</h4>

<p>Next I wanted to try serving the data through MapServer, and viewing/editing the data in Quantum GIS or MapInfo.&nbsp; MapServer and QGIS require the “geometry_columns” and “spatial_ref_sys” tables, and the easiest way to create these is to push some data in with ogr2ogr.</p>

<p>So I tried this from one of our London servers. I loaded a few hundred linear features, but got four errors like “INSERT command for new feature failed… incorrect syntax…”. On checking the results, a) while the geometries had loaded fine, all the attribute fields were NULL, and b) the spatial_ref_sys table was empty.&nbsp; I had four attribute fields, so my guess is that each syntax error was one field update across all records using a join on the incoming data.</p>

<p>Next I hooked up a MapServer WMS to my SQL Azure tables. MapServer had no trouble connecting to my Azure tables, but initially wouldn’t draw anything…&nbsp; on investigating I found that ogr2ogr had not correctly set the SRID of the geometries, so I had to update them “manually” with:</p>

<pre class="csharpcode">
<span class="kwrd">update</span> esdm_waters <span class="kwrd">set</span> ogr_geometry.STSrid = 4326</pre>
<style type="text/css">.csharpcode, .csharpcode pre
{
	font-size: small;
	color: black;
	font-family: consolas, "Courier New", courier, monospace;
	background-color: #ffffff;
	/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt 
{
	background-color: #f4f4f4;
	width: 100%;
	margin: 0em;
}
.csharpcode .lnum { color: #606060; }
</style>
<p>(which, incidentally, took 1 minute 52 seconds for 788 rows, demonstrating that updating an Azure database ain’t quick).</p>

<p>Clearly OGR needs a little work to do on SQL Azure support, but this was enough to get me going.</p>

<p>The WMS worked, so to challenge it a little more I set up another WMS onto a table with about 8000 sub-tidal habitat polygons, showing red in this OpenLayers map:</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_2.png"><img alt="MapServer WMS from SQL Azure table" border="0" height="127" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="MapServer WMS from SQL Azure table" width="244" />&nbsp;</a></p>

<p>And here showing the rivers WMS as well (blue lines):</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_4.png"><img alt="MapServer WMS from SQL Azure table" border="0" height="127" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_1.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="MapServer WMS from SQL Azure table" width="244" />&nbsp;</a></p>

<p>Performance of these two layers was perfectly acceptable, though actually about 100% slower than the same WMS working from a local SQL Server 2008 Express instance. It would be interesting to see what it could do with MapServer also running on Azure.</p>

<p>I also tried a WFS – again, performance was acceptable though significantly slower than running with a local SQL instance (though I didn’t do strict comparisons).</p>

<h4>Quantum GIS and SQL Azure</h4>

<p>I found this really helpful blog post to get started: <a href="http://woostuff.wordpress.com/2011/03/13/opening-ms-sql-spatial-in-qgis/" title="http://woostuff.wordpress.com/2011/03/13/opening-ms-sql-spatial-in-qgis/">http://woostuff.wordpress.com/2011/03/13/opening-ms-sql-spatial-in-qgis/</a></p>

<p>Adding the layers requires some python script, but I’m sure someone will write a GUI plugin for QGIS soon.</p>

<p>First make sure you are running a version of QGIS that has GDAL/OGR 1.8 libraries – I’m on 1.7.0 which is OK.</p>

<p>Open the Plugins menu &gt; Python Console</p>

<p>and type in these two commands (modified for your data):</p>

<pre class="csharpcode">
uri = <span class="str">"MSSQL:server={serverName};database={databaseName};tables={tableName};trusted_connection=yes"</span>
qgis.utils.iface.addVectorLayer(uri,<span class="str">'{yourLayerNameHere}'</span>,<span class="str">'ogr'</span>)
</pre>
<style type="text/css">.csharpcode, .csharpcode pre
{
	font-size: small;
	color: black;
	font-family: consolas, "Courier New", courier, monospace;
	background-color: #ffffff;
	/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt 
{
	background-color: #f4f4f4;
	width: 100%;
	margin: 0em;
}
.csharpcode .lnum { color: #606060; }
</style>
<p>In my case:</p>

<pre class="csharpcode">
uri = <span class="str">"MSSQL:server=tcp:a0awtn7gg9.database.windows.net;database=ArcSquirrel1;tables=esdm_waters;Uid=exegesis@a0awtn7gg9;Pwd=myPassword;"</span>
qgis.utils.iface.addVectorLayer(uri,<span class="rem">'Rivers','ogr')</span>
</pre>

<pre class="csharpcode">
Without any fuss my rivers dataset appeared:</pre>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_14.png"><img alt="SQL Azure table in Quantum GIS" border="0" height="237" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_6.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="SQL Azure table in Quantum GIS" width="244" />&nbsp;</a></p>

<p>And I can create and edit features and their attributes, here adding a test line off the Hebrides:</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_16.png"><img alt="Editing SQL Azure features in QGIS" border="0" height="237" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_7.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="Editing SQL Azure features in QGIS" width="244" />&nbsp;</a></p>

<p>And to prove it, here’s my new feature in OpenLayers…</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_18.png"><img alt="Edited features from QGIS and SQL Azure" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_8.png" style="margin: 0px; width: 244px; height: 127px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="Edited features from QGIS and SQL Azure" />&nbsp;</a></p>

<p>And finally, here we have the habitat polygons (EPSG:27700) colour coded by habitat type, as well as the rivers (EPSG:4326), in QGIS. Polygon editing also worked fine, once I’d worked out why my new polygons kept vanishing (doh, classified symbols).</p>

<p><a href="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_20.png"><img alt="SQL Azure polygons thematically mapped in QGIS" border="0" height="237" src="https://www.esdm.co.uk/Data/Sites/1/Windows-Live-Writer/Some-jottings-on-SQL-Azure-and-GIS_5C47/image_thumb_9.png" style="padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;" title="SQL Azure polygons thematically mapped in QGIS" width="244" />&nbsp;</a></p>

<p>Lots to think about, and I look forward to editing the same data with ArcGIS and ArcSquirrel soon…</p>
<br /><a href='https://www.esdm.co.uk/some-jottings-on-sql-azure-and-gis'>Crispin Flower</a>&nbsp;&nbsp;<a href='https://www.esdm.co.uk/some-jottings-on-sql-azure-and-gis'>...</a>]]></description>
      <link>https://www.esdm.co.uk/some-jottings-on-sql-azure-and-gis</link>
      <author>crispin.flower@idoxgroup.com (Crispin Flower)</author>
      <comments>https://www.esdm.co.uk/some-jottings-on-sql-azure-and-gis</comments>
      <guid isPermaLink="true">https://www.esdm.co.uk/some-jottings-on-sql-azure-and-gis</guid>
      <pubDate>Sat, 19 Nov 2011 11:42:00 GMT</pubDate>
    </item>
  </channel>
</rss>