Friday, October 16, 2009
Using GRASS to fix topology
Started trying to use GRASS to solve some of the more difficult problems that I can't fix in PostGIS. Here's one of them. I've imported a shapefile of country borders and am trying to fix the topology. The screenshot shows a country border with a bunch of "x" marks that I *think* represent centroids. I'd like to take them out, and just have one, long, continuous line at the country border. Not sure how to do that yet.
Wednesday, October 14, 2009
My take on commercial vs. open source GIS
Was alerted to a posting on the LinkedIn OGC group that seriously chapped my hide. A person had asked for advice regarding whether they should consider Open Source as a viable alternative to commercial software. You can read the original post at this link.
Here's my response.
"GIS" is now a very broad topic, and means different things to different people. You didn't specify what type of work you needed to do, so I'll try to address a few major topics that I encounter routinely:
- data conversion and creation,
- spatial analysis,
- server based data storage and access,
- desktop visualization,
- paper map production,
- web-based static maps,
- web-based dynamic maps,
and finally a weird one,
- reprojection to and from local projections from around the world.
- Data conversion and creation:
Hands down I prefer OS for this. Although I admit that I have encountered difficulties with CAD data that was created in bleeding edge commercial software. Usually this is because the OS development team has had to reverse engineer a new data format. I feel that there is far greater ability to work with data programatically using OS tools, and in my experience these tools suffer fewer failures when working with very large data sets. Having said that, if all I ever did was convert data, all day, every day, I probably would buy FME by Safe Software. They support more formats than I'm even aware of, and make it (relatively) easy to setup automated conversion pipelines. I don't need that degree of interoperability, and I use GDAL for practically every task that falls in this category.
- Spatial Analysis:
I feel that OS can do most spatial analysis quite capably. I'm a huge fan of doing vector based spatial analysis using SQL in PostGIS. I do 90% of my analysis this way. For more complicated topological analysis, as well as raster analysis, GRASS works extremely well. Having said that, my hands down favorite tool for analysis remains ArcMap. I am still much, much, faster in ArcMap, and I find that it is much easier to see the results of a particular operation with it. However, I rarely use it, as I can't run it on my laptop, and we can't afford another license anyhow.
- Server-based data storage and access:
Unless you have already heavily invested in commercial systems, or require database storage of raster data, OS (PostGIS specifically) wins in my book. If your entire workflow is already based on using many ArcGIS-equipped workstations which all connect to an Oracle Enterprise DB, I would be hesitant about deciding to convert to OS. Not because OS can't do it (except for database raster storage), but because there will be so much work in converting both the workflow, and the storage system itself. I understand from others that this is getting better - ESRI now supports PostGIS via SDE for example - but it's not without its headaches. Database storage of rasters is essentially non-existent in OS. There are all sorts of "solutions" talked about, but none of them are great, and frankly it's a glaring hole in OS.
- Desktop visualization:
Depends. If you want to quickly see what a data set looks like, or style several layers together, OS is great. It meets all of my needs on a daily basis. Then again, I think the grayscale display of 32-bit data in OpenEV is fantastic. There definitely is a lack of refinement in this area, but I'm hesitant to say OS isn't as good, mostly because I wonder just how sophisticated it needs to be?
- Paper Map Productionn:
For the frequent production of "casual maps", I think commercial software is still much better. I say "casual" because if you're making professional quality maps, I doubt you're going to just use a commercial GIS package. More than likely, you are also going to invest in a commercial graphics application as well. For "casual" paper maps, it's alot easier to use ArcGIS. Most of the time the results look pretty good, although the PDF engine is terrible and anything can happen. Personally, I use MapServer and GIMP to create paper maps with OS tools. I think they look pretty good, but they take more work than they should.
- Static web maps:
You're absolutely silly if you use anything but one of the OS tools for this. They can create lovely images - better in some cases than commercial products - and the only expense is the setup and configuration time.
- Dynamic and "slippy" web maps:
And now we get to what has everyone up in arms these days - everyone wants a mashup. Everyone is getting into this market, and everyone touts the capabilities of their solution at the expense of all others. I think that again it depends on what your goals are. If you want to create a web-based tool for spatial analysis, then I think the commercial offerings provide more "all in one" capability. If on the other hand, your goal is to display single or multiple styled layers, to allow for feature Identification and Selection (ESRI definition intended) - either by map click or attribute search, then there is no reason to use commercial software. In fact, I actually think that for these situations OS can both look and perform better. There are even OS base maps now that rival any of the commercial ones. I think there is a HUGE (note use of CAPS) market for, "3 vector layers and one image" on a web map, and I'm ecstatic to hear that a major vendor of commercial software doesn't think so.
- Reprojection support:
This frankly is an edge category, but's it's caused me problems numerous times. I suspect most people won't care though. OS software supports hundreds of common (and some not so common) projections via the PROJ library. Not only that, but it is easy to create your own custom projection as well, to display a specific area exactly how'you'd like. However, it does not deal with the reprojection of certain ones very well, especially those which use old, grid-based datums. Commercial software definitely still does a far better job at doing this.
Finally, a couple general observations about OS vs. commercial software. One of the major differences between commercial and OS applications is that in general OS applications do not try to address every conceivable situation you might possibly encounter in your work. Instead, they tend to focus on doing a few things extremely well, then rely on the ability to access other tools when the need arises. For example, Quantum GIS does a good job of displaying raster and vector layers, but relies on the ability to access GRASS in order to do more complex spatial analysis. If you are looking for a single solution for everything, commercial software with its multitude of extensions might be for you.
Support is also a good topic worth mentioning. In my experience, the people who like and insist on support contracts (aside from the vendors, of course), are the ones who never actually use the software. It's the ones buying the software that want it, and they view it as insurance that if something goes wrong with their very expensive purchase, they will get support. And usually there is a Service Level Agreement (SLA) that specifies they will get support in a timely manner, the amount of time depending on whether they are "Gold" or "Platinum" contract holders. But an SLA doesn't guarantee the quality of support - it can't. It merely ensures that the customer's problem will be dealt with in whatever support process the vendor has in place. Some companies are great, some are not, and it has nothing to do with how much money you paid for your support contract. In general the support given to OS applications by the developers and users of the product is as good, OR BETTER (note use of CAPS for emphasis) than any I have received via contract. Support extends beyond just staffing a help desk and charging a monthly maintenance/support fee though. It includes documentation of the product, and rapid bug fixes for critical problems.
Documentation for OS applications can sometimes be tough to find, and the quality is variable. There definitely is documentation out there, but sometimes it takes more effort than it should to find, and then it's not always easily digestible. You won't find the Help system that's in ArcGIS, that's for sure. But you will find many well-written Wiki's that cover most topics, and very responsive user lists to answer specific questions. Bug fixes and feature enhancements are far more rapid in the OS world, and tend to be in response to problems and requests reported by users. I think it's Documentation can sometimes be tough to find, this is true. There definitely is good documentation out there, but it is sometimes hard to find, and not easily digestible. You won't find the Help system that's in ArcGIS, for sure. But you will find many well-written Wiki's that cover most topics, and very responsive user lists to answer specific questions.a function of the development process that enables this. The bug tracking systems are transparent, and at any time you can go take a look at what's being done to resolve your problem, or to see what problems exist in a specific version. And lets not forget that "money talks" in the OS world as well. You absolutely need to be able to connect to that new/old/weird/unsupported database with your OS app, and you need it done NOW, and are willing to pay $5000 for the priviledge? There's a really good chance someone in the development community will do it for you.
So, to wrap this thing up, Open Source GIS software is just like anything else in life, it has both good and bad points. The same is true of commercial software as well. I don't think you can generalize across the entire category of uses with simple blanket statements. Identify what specific uses you need to address, then ask the question again. You'll probably get different answers for each one.
Best of luck.
Here's my response.
"GIS" is now a very broad topic, and means different things to different people. You didn't specify what type of work you needed to do, so I'll try to address a few major topics that I encounter routinely:
- data conversion and creation,
- spatial analysis,
- server based data storage and access,
- desktop visualization,
- paper map production,
- web-based static maps,
- web-based dynamic maps,
and finally a weird one,
- reprojection to and from local projections from around the world.
- Data conversion and creation:
Hands down I prefer OS for this. Although I admit that I have encountered difficulties with CAD data that was created in bleeding edge commercial software. Usually this is because the OS development team has had to reverse engineer a new data format. I feel that there is far greater ability to work with data programatically using OS tools, and in my experience these tools suffer fewer failures when working with very large data sets. Having said that, if all I ever did was convert data, all day, every day, I probably would buy FME by Safe Software. They support more formats than I'm even aware of, and make it (relatively) easy to setup automated conversion pipelines. I don't need that degree of interoperability, and I use GDAL for practically every task that falls in this category.
- Spatial Analysis:
I feel that OS can do most spatial analysis quite capably. I'm a huge fan of doing vector based spatial analysis using SQL in PostGIS. I do 90% of my analysis this way. For more complicated topological analysis, as well as raster analysis, GRASS works extremely well. Having said that, my hands down favorite tool for analysis remains ArcMap. I am still much, much, faster in ArcMap, and I find that it is much easier to see the results of a particular operation with it. However, I rarely use it, as I can't run it on my laptop, and we can't afford another license anyhow.
- Server-based data storage and access:
Unless you have already heavily invested in commercial systems, or require database storage of raster data, OS (PostGIS specifically) wins in my book. If your entire workflow is already based on using many ArcGIS-equipped workstations which all connect to an Oracle Enterprise DB, I would be hesitant about deciding to convert to OS. Not because OS can't do it (except for database raster storage), but because there will be so much work in converting both the workflow, and the storage system itself. I understand from others that this is getting better - ESRI now supports PostGIS via SDE for example - but it's not without its headaches. Database storage of rasters is essentially non-existent in OS. There are all sorts of "solutions" talked about, but none of them are great, and frankly it's a glaring hole in OS.
- Desktop visualization:
Depends. If you want to quickly see what a data set looks like, or style several layers together, OS is great. It meets all of my needs on a daily basis. Then again, I think the grayscale display of 32-bit data in OpenEV is fantastic. There definitely is a lack of refinement in this area, but I'm hesitant to say OS isn't as good, mostly because I wonder just how sophisticated it needs to be?
- Paper Map Productionn:
For the frequent production of "casual maps", I think commercial software is still much better. I say "casual" because if you're making professional quality maps, I doubt you're going to just use a commercial GIS package. More than likely, you are also going to invest in a commercial graphics application as well. For "casual" paper maps, it's alot easier to use ArcGIS. Most of the time the results look pretty good, although the PDF engine is terrible and anything can happen. Personally, I use MapServer and GIMP to create paper maps with OS tools. I think they look pretty good, but they take more work than they should.
- Static web maps:
You're absolutely silly if you use anything but one of the OS tools for this. They can create lovely images - better in some cases than commercial products - and the only expense is the setup and configuration time.
- Dynamic and "slippy" web maps:
And now we get to what has everyone up in arms these days - everyone wants a mashup. Everyone is getting into this market, and everyone touts the capabilities of their solution at the expense of all others. I think that again it depends on what your goals are. If you want to create a web-based tool for spatial analysis, then I think the commercial offerings provide more "all in one" capability. If on the other hand, your goal is to display single or multiple styled layers, to allow for feature Identification and Selection (ESRI definition intended) - either by map click or attribute search, then there is no reason to use commercial software. In fact, I actually think that for these situations OS can both look and perform better. There are even OS base maps now that rival any of the commercial ones. I think there is a HUGE (note use of CAPS) market for, "3 vector layers and one image" on a web map, and I'm ecstatic to hear that a major vendor of commercial software doesn't think so.
- Reprojection support:
This frankly is an edge category, but's it's caused me problems numerous times. I suspect most people won't care though. OS software supports hundreds of common (and some not so common) projections via the PROJ library. Not only that, but it is easy to create your own custom projection as well, to display a specific area exactly how'you'd like. However, it does not deal with the reprojection of certain ones very well, especially those which use old, grid-based datums. Commercial software definitely still does a far better job at doing this.
Finally, a couple general observations about OS vs. commercial software. One of the major differences between commercial and OS applications is that in general OS applications do not try to address every conceivable situation you might possibly encounter in your work. Instead, they tend to focus on doing a few things extremely well, then rely on the ability to access other tools when the need arises. For example, Quantum GIS does a good job of displaying raster and vector layers, but relies on the ability to access GRASS in order to do more complex spatial analysis. If you are looking for a single solution for everything, commercial software with its multitude of extensions might be for you.
Support is also a good topic worth mentioning. In my experience, the people who like and insist on support contracts (aside from the vendors, of course), are the ones who never actually use the software. It's the ones buying the software that want it, and they view it as insurance that if something goes wrong with their very expensive purchase, they will get support. And usually there is a Service Level Agreement (SLA) that specifies they will get support in a timely manner, the amount of time depending on whether they are "Gold" or "Platinum" contract holders. But an SLA doesn't guarantee the quality of support - it can't. It merely ensures that the customer's problem will be dealt with in whatever support process the vendor has in place. Some companies are great, some are not, and it has nothing to do with how much money you paid for your support contract. In general the support given to OS applications by the developers and users of the product is as good, OR BETTER (note use of CAPS for emphasis) than any I have received via contract. Support extends beyond just staffing a help desk and charging a monthly maintenance/support fee though. It includes documentation of the product, and rapid bug fixes for critical problems.
Documentation for OS applications can sometimes be tough to find, and the quality is variable. There definitely is documentation out there, but sometimes it takes more effort than it should to find, and then it's not always easily digestible. You won't find the Help system that's in ArcGIS, that's for sure. But you will find many well-written Wiki's that cover most topics, and very responsive user lists to answer specific questions. Bug fixes and feature enhancements are far more rapid in the OS world, and tend to be in response to problems and requests reported by users. I think it's Documentation can sometimes be tough to find, this is true. There definitely is good documentation out there, but it is sometimes hard to find, and not easily digestible. You won't find the Help system that's in ArcGIS, for sure. But you will find many well-written Wiki's that cover most topics, and very responsive user lists to answer specific questions.a function of the development process that enables this. The bug tracking systems are transparent, and at any time you can go take a look at what's being done to resolve your problem, or to see what problems exist in a specific version. And lets not forget that "money talks" in the OS world as well. You absolutely need to be able to connect to that new/old/weird/unsupported database with your OS app, and you need it done NOW, and are willing to pay $5000 for the priviledge? There's a really good chance someone in the development community will do it for you.
So, to wrap this thing up, Open Source GIS software is just like anything else in life, it has both good and bad points. The same is true of commercial software as well. I don't think you can generalize across the entire category of uses with simple blanket statements. Identify what specific uses you need to address, then ask the question again. You'll probably get different answers for each one.
Best of luck.
Subscribe to:
Posts (Atom)