Jump to content
  • 0
Tim

FME Transformers

Question

21 answers to this question

Recommended Posts

  • 2

@TasGeodesist: Yes, glad you noticed! As per the updated note, the NTv2 grid is now available in FME 2018.0 Betas from Build 18240.

While we had thought the NTv2 grid would supersede the similarity transform in all cases, it turns out users will have to choose between the conformal transform or the combined conformal + distortion transform. This is discussed elsewhere in this forum, e.g.: http://gda2020.invisionzone.com/topic/20-application-of-ntv2-distortion-grid/. My sense is that the best summary of this issue is the technical manuals: http://www.icsm.gov.au/gda2020/tech.html

FME now supports both options. However, in the moments after shipping our Build 18240, it occurred to us that the names of those options should clearly reflect the conformal vs. conformal-plus-distortion choice; we've made that update in a subsequent build that should be available in the coming days.

Share this post


Link to post
Share on other sites
  • 1

Thanks Alex! I've signed in and clicked the "watch" button there, so we'll be ready to act when it's available.

We especially appreciate the legal clarity - that's likely to save us a few back-and-forth round trips.

Regards,

Paul

Share this post


Link to post
Share on other sites
  • 0

Hi Tim,

You will be able to use FME to transform between GDA94 and GDA2020 when the transformation parameters (7 parameter similarity) and / or NTv2 grid are released.  

The 7 parameters aren't far away but the NTv2 grid is not likely to be available until March / April.  It will be well publicised when they are available on the ICSM and GA websites.

It is possible to manually add the transformation information to FME (define your own transformation) and there will be a advice provided on this once the transformations are available.

However, FME is one of the many software providers Geoscience Australia is in conversation with about adding the transformation as part of a standard install, via either a patch or upgrade, to fast-track the uptake of the new datum.

  • Upvote 3

Share this post


Link to post
Share on other sites
  • 0

Gustavo - Absolutely this is something FME can do. I spoke with our Experts team and they provided a link to a tutorial very close to your scenario: https://knowledge.safe.com/articles/1003/mapinfo-tab-to-esri-shape.html The best launching point for getting help about FME is here - https://www.safe.com/support/ - but if you get stuck along the way, I would be happy to help.

For the larger group - I'm pleased to share that FME 2017.1, released August 1st, includes full support for GDA2020 as per our plan linked above. The only thing missing is the NTv2 grid, and we are standing by to integrate that very shortly after its release. For those already using FME, you can download that build here: https://www.safe.com/support/support-resources/fme-downloads/

Share this post


Link to post
Share on other sites
  • 0


Thanks for this advice Paul.

The national GDA94-GDA2020 NTv2 distortion grid is now very close to be being finalised. Software providers will be notified directly when it is but the grids will be freely accessible via the method described by Alex previously. 

The ICSM website - http://www.icsm.gov.au/gda/grids.html will also provide background information / resources on all the Australian grids.

Share this post


Link to post
Share on other sites
  • 0

Hi - bit late to the party here, but..

FME handles shapefile transformation (individually or whole directory - with a Workspace Runner) very well using the CsmapReprojector.

Unfortunately, I haven't had the same success with MapInfo TAB files - FME generates a warning "MapInfo datum `1028' unrecognized. Coordinate system metadata will be written without datum" - the file is output in EPSG 32756 [WGS84 / UTM Zone 56S].

SAFE support have confirmed that this is still the case (23/11/18) and have filed a problem report for the developers to work on.

 

Share this post


Link to post
Share on other sites
  • 0
Quote

Unfortunately, I haven't had the same success with MapInfo TAB files - FME generates a warning "MapInfo datum `1028' unrecognized. Coordinate system metadata will be written without datum" - the file is output in EPSG 32756 [WGS84 / UTM Zone 56S].

There are four writers in FME for MapInfo:

- MIF (for the MIF/MID interchange format) - This works.

- MITAB (for TAB files, via a 3rd party library) - This works.

- MAPINFO (for TAB files, via the official MFAL library) - This does not work. Data is written incorrectly as WGS84.

- MAPINFO_EXTENDED (for TAB files, via the official EFAL library) - This does not work. Data is written incorrectly as non-earth miles.

We're standing by to integrate an updated version of the EFAL library as soon as it's available, and are discussion with Pitney Bowes as to the appropriate next steps for MFAL.

Quote

 SAFE support have confirmed that this is still the case (23/11/18) and have filed a problem report for the developers to work on.

This is true. Our internal identifier for this issue is FMEENGINE-57578.

Share this post


Link to post
Share on other sites
  • 0
8 hours ago, Paul Nalos said:

There are four writers in FME for MapInfo:

- MIF (for the MIF/MID interchange format) - This works.

- MITAB (for TAB files, via a 3rd party library) - This works. 

- MAPINFO (for TAB files, via the official MFAL library) - This does not work. Data is written incorrectly as WGS84.

- MAPINFO_EXTENDED (for TAB files, via the official EFAL library) - This does not work. Data is written incorrectly as non-earth miles.

We're standing by to integrate an updated version of the EFAL library as soon as it's available, and are discussion with Pitney Bowes as to the appropriate next steps for MFAL.

This is true. Our internal identifier for this issue is FMEENGINE-57578.

As per the thread directly on the FME site, we have no luck with the dot point 2 above " MITAB (for TAB files, via a 3rd party library) - This works. ", we still get the same error "assuming default...."

https://knowledge.safe.com/questions/67350/why-do-i-get-warning-and-assumed-default-when-tran.html?childToView=83306#comment-83306

Regards,

Tony

Share this post


Link to post
Share on other sites
  • 0
Quote

As per the thread directly on the FME site, we have no luck with the dot point 2 above " MITAB (for TAB files, via a 3rd party library) - This works. ", we still get the same error "assuming default....

We're still investigating - there are more details in the linked thread - but we have one more clue: While FME 2018.0 added support for GDA2020, the MapInfo portion was only added in FME 2018.1. So to successfully write GDA2020 coordinate systems to MapInfo TAB at this time, you need to do two things: (1) Use FME 2018.1.0.0 or newer, and (2) Use the MITAB writer (and not the MAPINFO or MAPINFO_EXTENDED ones).

I'll update this thread when we learn more, or when we address (2).

Share this post


Link to post
Share on other sites
  • 0

Latest updates on that Q&A page appear to have addressed (not entirely fixed yet) the issue.  

I have asked for confirmation that the fixes (given the Projection Bounds warning) work for all geometries, not just points. My single polygon (almost rectangular region) test dataset ended up as butterfly geometry.

Share this post


Link to post
Share on other sites
  • 0
2 hours ago, Mark said:

 I have asked for confirmation that the fixes (given the Projection Bounds warning) work for all geometries, not just points. My single polygon (almost rectangular region) test dataset ended up as butterfly geometry.

I did an experiment just now with a 1m-by-1m square, and it appeared correctly in MapInfo. I suggest you follow up with our support team at https://www.safe.com/support/report-a-problem/ with your sample case (or perhaps you're already in touch with support). That said, if you run into any bumps in the process, feel free to reach out to me directly at paul.nalos@safe.com.

Share this post


Link to post
Share on other sites
  • 0

Hi Paul,

I have been working with Andrea from your support team and have a working workspace (and associated runner) which has allowed me to transform a directory of TAB files.

By converting the results to SHP and comparing with previously transformed data (various mechanisms) in ArcGIS I can see that the results agree (within 5mm). 

I am working with PB to acquire a new trial version of MapInfo to finalise my testing, but that's my problem.

Thanks for your help on this.

Mark

Share this post


Link to post
Share on other sites
  • 0

@Mark: You're very welcome. Andrea and I have been working closely on this; I'll pass on your thanks!

(As far as I know, we've not been able to reproduce your butterfly-geometry problem - although it sounds like a way-too-corse bounds grid; did you find a way to get past that issue?)

Share this post


Link to post
Share on other sites
  • 0

@Paul Nalos After much testing, I believe the odd-geometry was caused by me trying to set bounds (in response to the invalid bounds type warning - that you also got) when this isn't necessary - it hasn't happened since.

Thanks again for the assistance.

Share this post


Link to post
Share on other sites
  • 0

@Mark: That makes great sense - thank you. Basically you get ~4 billion grid lines between your min/max bounds in x and y. If your data is more fine than that, it gets shattered (or moved) in various "exciting" ways as it snaps to the grid (including polygons turning into butterflies). Without hardcoded bounds, FME uses the bounds of the data you write - which is excellent for precision issues (the resulting grid is usually very fine) - but not great if you want to edit the data later to have a wider extent. If requested, we could guess reasonable bounds for each GDA2020 coordinate system, add 10-20%, and hardcode those, avoiding this last problem. In any case, for this level of detail, best to use our normal support channels, but I'm certainly available if other avenues fail.

Share this post


Link to post
Share on other sites
  • 0

Hi Paul,

Adding a (user-specifiable) buffer % sounds like a reasonable feature.  An additional option could be to use the bounds of another existing dataset.

Cheers,

Mark

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×