Adding field with filename when merging shapefiles with ogr2ogr

Question

I'm merging some shapefiles and I had some problems doing so inside QGIS, so i'm using ogr2ogr directly. I'm doing this (in a batch):

ogr2ogr -overwrite %destination% %n1%
ogr2ogr -update -append %destination% %n2% -nln all_new
ogr2ogr -update -append %destination% %n3% -nln all_new
ogr2ogr -update -append %destination% %n4% -nln all_new

It works fine, but now I need to have in the resulting shapefile, a field with the names of the original shapefiles I merged. Doesn't sound very difficult, but I'm not managing to do it.

Can anyone help?

PolyGeo · Accepted Answer · 2023-01-07 00:32:07Z

With small scripting it would be doable. With something like following you should be able to add column to a shapefile in all shapefiles in a folder, and merge them to merged.shp file

for %f in (*.shp) do (
  ogrinfo %f -sql "ALTER TABLE %f ADD COLUMN filename character(15)"
  ogrinfo %f -sql "UPDATE TABLE %f filename = '%f'"
  ogr2ogr -update -append merged.shp %f -f “esri shapefile” -nln merge 
)

The same as a Bash script, with some changes to make it work:

for f in *.shp
do 
  base=${f%.shp}
  ogrinfo $f -sql "ALTER TABLE $base ADD COLUMN filename character(15)"
  ogrinfo $f -dialect SQLite -sql "UPDATE $base SET filename = '$base'"
  ogr2ogr -update -append merged.shp $f
done

capooti · Accepted Answer · 2012-03-23 13:14:32Z

10

I would use the -sql option, and import the shapefile in the following way:

ogr2ogr -update -append %destination% %n2% -sql 'SELECT "%n2%" as SHAPE_ORIG, field1, field2, ... FROM %n2%'

edited Mar 23, 2012 at 13:14

answered Mar 23, 2012 at 13:07

capooti

2,70821 silver badges22 bronze badges

Add a comment |

PolyGeo · Accepted Answer · 2023-01-07 00:32:59Z

there are some ways for merging shapefiles.

if you want to merge layers as a one layer, you can use MMqgis tools for merging...

mmqgis

if you want to merge all shapefiles under a folder, you can use DARREN COPE simple code here.

mkdir merged
for %f in (*.shp) do (
if not exist merged\merged.shp (
ogr2ogr -f “esri shapefile” merged\merged.shp %f) else (
ogr2ogr -f “esri shapefile” -update -append merged\merged.shp %f -nln Merged )
)

beside this can use GeoMerge free tool for merging lots of file but dont forget to consider your file size for workin with it.

and adding attribute to shapefile @dango directon is good. you can use layer.CreateField(field_name) for creating a new column which is populated from

import os
shapeFileName = os.path.splitext("your_shape_file_path")[0]

PolyGeo · Accepted Answer · 2023-01-07 00:33:50Z

Here is how I accomplished this problem by using a Python script to daisy-chain several ogr2ogr instructions together. You could easily convert it to a batch script, basically I just concatenate together ogr2ogr instructions (cmd), then execute them by calling os.system(cmd), passing-in the ogr2ogr command I concatenated together.

The secret weapon is (as @capooti demonstrated) applying OGR_SQL to impose the filename as a constant value of the source dataset you are appending into your merge result.

In my example, the -sql flag handles this, in the code it's like this:

-sql "SELECT \'' + filename + '\' AS filename, * FROM ' + filenameNoExt + '"'

But that's confusing to read because I need to apply single quotes and double quotes in the resulting concatenation. To do that I have to escape the single quotes (i.e. ') to use them "for real". So for readability, it helps to see it without variables and escape sequences. If you pretend the filename was "roads1" for a particular iteration, the resulting concatenation would look like this in the ogr2ogr sentence:

-sql "SELECT 'roads1.shp' AS filename, * FROM roads1"

This .py script is an amalgamation of three tricks I stole from matt wilkie (an empty, clone of a shapefile), j03lar50n (adding a column to a shapefile using ogrinfo and ogr_sql), and capooti (using ogr_sql to impose a fixed column value on all records in a shapefile). So here's the full script:

# merge_shps.py
import os    

path = "D:/GIS/01_tutorials/ND_Roads/extracted"  # path to your folder of .shp files
merge = "merge_filename"                         # this will be the name of your merged result

directory = os.listdir(path)

count = 0
for filename in directory:
    if ".SHP" in filename.upper() and not ".XML" in filename.upper():
        
        # On the first pass, create a clone and add the filename column.
        if count == 0:
            # Make a clone (matt wilkie)..
            cmd = 'ogr2ogr ' + path + '/' + merge + '.shp ' + path + '/' + filename + ' -where "FID < 0"'
            os.system(cmd)
            
            # Add the field (j03lar50n)..
            cmd = 'ogrinfo ' + path + '/' + merge + '.shp -sql "ALTER TABLE ' + merge + ' ADD COLUMN filename character(50)"'
            os.system(cmd)
        
        # Now populate the data (capooti)..
        print "Merging: " + str(filename)
        
        # You'll need the filename without the .shp extension for the OGR_SQL..
        filenameNoExt = filename.replace(".shp","")
        
        cmd = 'ogr2ogr -f "esri shapefile" -update -append ' + \
                path + '/' + merge + '.shp ' + \
                path + '/' + filename + \
                ' -sql "SELECT \'' + filename + '\' AS filename, * FROM ' + filenameNoExt + '"'
        
        # Uncomment this line to spit the ogr2ogr sentence to the terminal..
        #print "\n" + cmd + "\n"
        
        os.system(cmd)
        
        count += 1

oeon · Accepted Answer · 2012-12-01 02:29:34Z

5

Add column with source filename from folder of shapefiles. Requires GDAL 1.10dev, my attempt to drop .shp extension isn't working - but overall, works. - I imagine it could be added to the lines that do merging with OGR.

for f in *.shp;

do

name=${f%.shp}

/Users/you/gdal_src/bin/ogrinfo $f -sql "ALTER TABLE $name ADD COLUMN filename character(21)"
/Users/you/gdal_src/bin/ogrinfo $f -dialect SQLite -sql "UPDATE $name SET filename = '$f'"
done;

answered Dec 1, 2012 at 2:29

oeon

5443 silver badges9 bronze badges

Add a comment |

PolyGeo · Accepted Answer · 2023-01-07 00:34:25Z

4

There is also ogrmerge:

ogrmerge.py -single -o merged.shp *.shp -src_layer_field_content {DS_BASENAME}

edited Jan 7, 2023 at 0:34

PolyGeo♦

65.2k29 gold badges109 silver badges340 bronze badges

answered Sep 15, 2019 at 10:58

Anatol

4753 silver badges11 bronze badges

+1 This is by far the simplest solution
– James
Commented Jun 22, 2023 at 6:06

Add a comment |

Ryan Garnett · Accepted Answer · 2013-06-07 19:03:56Z

1

Inside QGIS you can add the Merge Shapefile plugin. There is an option to "Add column with file name" enter image description here

answered Jun 7, 2013 at 19:03

Ryan Garnett

9,4898 gold badges62 silver badges107 bronze badges

Add a comment |

score 0 · Accepted Answer · 2016-07-29 19:55:14Z

A slightly modified version of JaaKL's answer. Note that the -append foo.shp and -nln foo need to match. Also, note the use of the SQLite dialect (GDAL apparently doesn't accept the keyword 'Update', so the SQLite dialect must be used instaed), and the absence of the keyword 'TABLE' after the word 'UPDATE' (not needed or accepted by SQLite).

for %%f in (*.shp) do (
  if not "%%f" == "merge.shp" (
    ogrinfo %%f -sql "ALTER TABLE %%~nf ADD COLUMN fname character(15)"
    ogrinfo %%f -dialect SQLite -sql "UPDATE %%~nf SET fname = '%%~nf'"
    ogr2ogr -update -append merge.shp %%f -f "ESRI SHAPEFILE" -nln merge 
  )
)

Stack Exchange Network

Adding field with filename when merging shapefiles with ogr2ogr

8 Answers 8

Not the answer you're looking for? Browse other questions tagged
gdal
ogr2ogr
or ask your own question.

Linked

Hot Network Questions

Adding field with filename when merging shapefiles with ogr2ogr

8 Answers 8

Not the answer you're looking for? Browse other questions tagged gdalogr2ogr or ask your own question.

Linked

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
gdal
ogr2ogr
or ask your own question.