Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Stainless]

I am looking at it.

The data is a lot lower resolution than the terrain data, but I think I can use it for the pallete map.

At least to identify water more accurately

[Stainless]

So looking at the data in detail, the bathometric SRTM file covers 40 degrees of longitude and 50 degrees of latitude, giving me a 120 by 120 array of data per 1 degree cell.

This means I have a 2 by 2 array per 1 minute texture tile.

I think this means I can accurately detect open water and coastline.

[Pursuivant]

Glad that the data sort of helped, even if it wasn't a perfect fix.

I've done a bit of further research on bathymetric data sets and data resolution levels are all over the place, with 1-2 meter resolutions for a few coastal areas to up to 30 arc seconds for more distant areas.

An alternate source of bathymetric data which might be helpful is:

https://www.bodc.ac.uk/data/hosted_data_systems/gebco_gridded_bathymetry_data/

I'll keep looking for better data sets, particularly for coastal areas, but I don't think that there will be any one data set which has everything. Detailed bathymetric maps for coastal areas might need to be a follow-up project for someone else to do, as long as you make it easy to port more detailed maps into the simulation.

[Stainless]

Current state of play of the terrain system.

• Generating height maps

I have written a tool to generate the height maps from SRTM data.

SRTM data is available in 1 degree arcs, so a single file covers 1 degree of latitude and 1 degree of longitude.

I load this data and convert the values within to signed shorts (16 bits of height) which give us a height range of 65536 M. ( roughly +- 32768 ).

Then I split this down to 1 second of arc. So a single SRTM data file gives us 60 by 60 height maps. Each height map is 1025 by 1025 and stored as a binary file not a bitmap. (A 1025 by 1025 bitmap when loaded would actually occupy 2048 by 2048 pixels in video ram on some graphics cards, obviously not a good thing ).

• Generating normal maps

This turned out to be very complex. I tried many different techniques and all of them produced visible artifacts. In the end I bought  a commercial tool and reverse engineered it. Now the normals are good enough. I basically generate a very simple vertex based normal per pixel and then apply a couple of layers of blurs on the top to get something that works.

The technique I originally used should have worked, it is still a puzzle to me why it did not. I generated face normals for all faces in the height map, and added this to a per-pixel accumulator. At the end I normalised these accumulators to create the normal.

Which produced horizontal lines across the terrain. Don't know why.

The generated normal map is stored as a 1024 by 1024 pixel bitmap and sampled in the shader.

• Generating type maps

Each 1 second of arc region can have 4 textures assigned to it. So I generate a type map where each byte of the pixel represents the blend factor for each of those textures.

On paper this should work well, but coming up with a good way of generating these type maps is proving difficult.

I am currently using 4 curves that overlap and calculating a set of values based on height, but that doesn't handle cliffs well. So I am going to swap to a system based on normals.

My plan is to have two regions. One below 6000 metres and one above.

Below 6000 I will blend between texture 0 and texture 1 based on height, then overwrite any pixels with a normal nearer horizontal with texture 2.

Above 6000 I will blend between 2 and 3 based on height then overwrite any pixels with a normal nearer horizontal with texture 2.

• Display

I have a streaming system in place which loads terrain patches from disk when needed. Based on a visibility check and a temporal filter.

Each 1025 by 1025 height map is scaled based on the latitude and longitude. Then the width and height are extended by 10 metres so they overlap with the next height map.

From this I generate a CLOD terrain patch. So only the visible parts are rendered and all have LODS applied.

In the shader I blend the pixel by the type map to get a base colour, then apply lighting.

At the moment the shader also clips the polygons against the ground plane not drawing anything below zero. I did this because the water shader draws a refracted version of the terrain for the region below the water line.

I am going to have to remove this as some parts of the world are dry and below zero altitude, sigh.

• Problems

• Water line : Blocky

The water line is too blocky. I need to change the type map so anything under water has all the blend values set to zero. Then I can use this in the water shader to generate foam or waves to hide this artifact.

• Water line : Location

SRTM data is taken from satellite data which sees the sea. (Groan). So open water has a height depending on the sea state at the time the data was gathered. I cannot find any GIS data accurate enough to automatically solve this problem.

 

• Missing data

Sometimes the SRTM data has holes in it. I cannot come up with a good way of automatically fixing these issues. Everything I have tried looks awful.



So guys please join this thread if you have any ideas or questions or sources of GIS data on anything that can help.

At the moment I cannot see any alternative to manually editing all the maps to fix the issues. Something I was desperately trying to avoid.

Ideally I would find a way of getting satellite imagery for each height map and using that as a detail texture, but there are all sorts of problems with that. Time period. Copyright. Alignment. The list goes on.

Any help gratefully accepted.

[ildifa]

    Water line : Location

SRTM data is taken from satellite data which sees the sea. (Groan). So open water has a height depending on the sea state at the time the data was gathered. I cannot find any GIS data accurate enough to automatically solve this problem.

Would the GIS data found in this website [https://www.naturalearthdata.com/downloads/10m-physical-vectors/] be useful for you?
I used the vector files (this in particular: https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_land.zip) in my work, to produce a sea/land raster mask. You would lose the position of almost all lakes, which are not present in the dataset, but I guess it's better than nothing...
But maybe for the lakes you can keep the SRTM data, as I imagine the waves are much smaller than in the open sea.

Hope to be of help, and good luck in all your wonderful work!

[Stainless]

I have looked at this data many times. I don't know, maybe it's just me being a perfectionist, but I cannot get on with it.

The basic problem is accuracy. I have height data at 1M , but the best they can offer me is 10M.

And it's just not accurate enough for me. I know the problem is me not the data 

For example.

This is the coastline data drawn over the height data, the two don't match.




So I tried the bathymetry data



Which as you can see is just generated from the coastline data

So I tried the land data



And it's just the reverse of the water data.

Sigh.

Maybe I can come up with some way of using the data, but at the moment it feels like that is a whole new project.

[Stainless]

That started the little grey cells.

I wondered if I was thinking about it in the wrong way. Instead of trying to get more accurate data, maybe I could use the data I have to generate the data I want.

So I grabbed a height map in paint.net and ran an outline filter on it twice. Once with a small radius and once with a larger radius.

I then combined these to create a new image





That's looking a lot closer

[ildifa]

Wow, that's a great idea!
I guess now you could use a "shallow sea" texture for the yellow outline, and preserve the position of beaches and cliffs. It also solves the problem of the non-zero height of the open sea, AND it's not blocky... Perfect!

[Stainless]

Okay a lot of changes.

Originally I stored the height maps as 16 bit signed integers, this just didn't give enough accuracy.

So I now calculate the height range and scale the 16 bit value to fit in that range.

The height is now

Code: [Select]

float height = (sample * heightscale) + minimumheight;
This gives each height map much more resolution.


I have also realised I am texturing the terrain all wrong. Instead of using the height to texture a pixel, we are going to need to be able to apply a hand edited texture, or a satellite image.

So I have changed the code to blend between a supplied texture and a rock texture based on slope. So cliffs will be textured properly.

This gives me the problem that we need textures for ever height map.

So I have written an app to test a system to auto-generate the textures for all height maps. A default texture that can be edited, replaced, or what ever.

It starts by loading the heights and storing them in a floating point texture so all the rest of the processes can be done in shaders.




Then I generate the type map based on those heights.



Then I pass it through a couple of blurs.



And then I use supplied textures to generate a default texture map.



The next stage is to generate the surf map.

[Stainless]

Doesn't look too bad as a starting point.





Desperately needs a better water shader and the surf line.

[Stainless]

Well I can generate sea shore textures.




They seem to be in the right place.




So now I just need to figure out how to render the foam.

[SAS~Ghost129er]

Perhaps like IL2? An alpha channel animated texture that's a border frame of land textured like water risen slightly above the shore and water..?

Sorry struggling to explain what I mean...