Terrain texturing

If you have any questions on programming, this is the place to ask them, whether you're a newbie or an experienced programmer. Discussion on programming in general is also welcome. We will help you with programming homework, but we will not do your work for you! Any porting requests must be made in Developmental Ideas.
Post Reply
b0b
DCEmu Fast Newbie
DCEmu Fast Newbie
Posts: 20
Joined: Thu Mar 15, 2007 12:04 pm
Has liked: 0
Been liked: 1 time

Terrain texturing

Post by b0b » Sun Jan 31, 2016 8:18 pm

Hi Everyone!


I need some programming support...

I'm implementing a terrain renderer. I'm using OpenGL on the PC side but I'm going PVR-only on the dreamcast. The terrain LOD system is mostly done... And now I'm trying to figure out how to properly texture this thing. I have like 4 different textures (the basic gras, rock, dirt, snow stuff) in mind which need blending (kinda slope based not just height based).

As far as I get it, you have to carefully sort your triangles in opaque or transparent lists. Once your are done with one you can only add stuff to the other one - or implement some kind of a buffer and save it for later. My terrain is rebuilt every frame (SOAR-algorithm based) so this is kinda difficult to manage. So the best way would be to render only opaque polygons as terrain polys. As the dreamcast doesn't have multitexture units I was thinking of two options here

1) pre-bake all textures and stream them from the cd-drive, like virtual-texturing
- I've read somewhere that one can expect 350-450kb/s from the cd drive, so that's not an option

2) generate texture-tiles on-the-fly by blending all the needed textures together, maybe also light-map them and then render them to a texture atlas

so I'm investigating option 2) here :)
I think I have a possible solution, but I'm stuck at the following:

as I understand it, it's only possible to render to a texture which is bigger then the screen size. so for 640x480 the texture should be 1024x512. that's bad :( the memory is already very limited.
Is it only a implementation issue in KOS? or is it possible to render something in to a 128x128,256x256,512x512 texture regardless of the renderer screen size? setting the perspective+projective matrix maybe?
Is it possible to render something to an exact part of the texture?

like I want to render in to block A here:

Code: Select all

0---------128---------512
|              |                 |
|              |                 |
|              |                 |
-------------------------
|              |                 |
|              |       A        |
|              |                 |
|              |                 |
-------------------------
Is it possible to copy the A part from one texture in an other? I mean is there a fastest way to do it or is it simply copying it manually row by row?
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has liked: 0
Been liked: 3 times

Re: Terrain texturing

Post by PH3NOM » Sun Jan 31, 2016 10:13 pm

Terrain is generally very easy; the height map r,g,b values are interpreted as x,y,z, and viola, 3D terrain.

I would advise use OpenGL instead of trying to use the PVR directly: the PVR does no 3D transformation for you /:
OpenGL supports Multi-Texture on DC in the current KOS, limited to 2 texture units enabled.
I wrote examples kos\examples\dreamcast\kgl\demos\multitexture-arrays and kos\examples\dreamcast\kgl\demos\multitexture-elements

Yes, using KOS, render-to-texture must be done at 640x480 into a 1024x512 texture.
You could potentially use clipping to only render to a certain part of that texture if you are only updating a small part of it...
b0b
DCEmu Fast Newbie
DCEmu Fast Newbie
Posts: 20
Joined: Thu Mar 15, 2007 12:04 pm
Has liked: 0
Been liked: 1 time

Re: Terrain texturing

Post by b0b » Mon Feb 01, 2016 3:05 pm

PH3NOM wrote:Terrain is generally very easy; the height map r,g,b values are interpreted as x,y,z, and viola, 3D terrain.
yeah, well, except it isn't if you are trying to render detailed terrain with good draw distance.
you have to implement some sort of LOD-system. nowadays on newer hardware you can do pretty much everything terrain related in shaders. on the dreamcast the SH4 is your only friend :(
there are still some papers (from around 2000-2002) around to make terrain LOD on the CPU. MisterDave here on the board seems to use the ROAM-algorithm for LOD for example (I'm trying another approach, but might also go back to ROAM if it's not good enough...)
PH3NOM wrote: I would advise use OpenGL instead of trying to use the PVR directly: the PVR does no 3D transformation for you /:
OpenGL supports Multi-Texture on DC in the current KOS, limited to 2 texture units enabled.
I wrote examples kos\examples\dreamcast\kgl\demos\multitexture-arrays and kos\examples\dreamcast\kgl\demos\multitexture-elements
yes, I know. I'm okay doing the transformations myself.
I'm aware of your work on KGL btw (and I honestly think it's great!), i read on these board regularly, just don't post much ;)
PH3NOM wrote: Yes, using KOS, render-to-texture must be done at 640x480 into a 1024x512 texture.
You could potentially use clipping to only render to a certain part of that texture if you are only updating a small part of it...
hmm... okay. I guess, I need to rethink how to properly use this big texture.
I might have an idea


I'm not sure if MisterDave reads this but I kindly hope he shows up in this thread
(given the topic might be interesting for him)
User avatar
MisterDave
DCEmu Freak
DCEmu Freak
Posts: 58
Joined: Mon Apr 08, 2013 1:16 pm
Has liked: 0
Been liked: 0

Re: Terrain texturing

Post by MisterDave » Tue Feb 02, 2016 6:44 am

Hi B0b

Just seen your post. For the terrain I'm using a greyscale 8-bit image which gives you 256 possible height values. This can be scaled up for more height, but will increase the visible step size unless you do some smoothing. A 16 bit heightmap can be used but that increases memory usage.

The approach I'm using for LoD is a fairly good for CPU usage but is likely not using the cache as efficient as I would like and also has the problem of gaps when the detail level changes. All things to fix in the future, but I need to look at a few other algorithms on vterrain.org to help inform this in addition to the WoW format.

Blending the textures isn't too difficult with an RGB map (with each channel representing the intensity of each texture if you don't need too many textures) and some blurring with GIMP. It's pretty straightforward to draw a base texture then a transparent one over the top based on the RGB map. I'm using a single texture atlas so only need to bind one physical texture, however, logical textures can't be repeated per poly in the distance for low LoD polys (U/V bigger than 1), so I've had to create some doubled textures at the same size of the normal ones (and so on).

I've used the RGB also to decide where my terrain foliage and rocks go which will be in an upcoming video in the next day or so.

Rendering to texture is next on the agenda for me for DoF. I did this achieve using a modification to the original KGL a couple of years back and am currently adapting my code to work on Phenom's new API as he uses a much more modern FBO approach as far as the wrapper is concerned.

Cheers,
Dave
b0b
DCEmu Fast Newbie
DCEmu Fast Newbie
Posts: 20
Joined: Thu Mar 15, 2007 12:04 pm
Has liked: 0
Been liked: 1 time

Re: Terrain texturing

Post by b0b » Tue Feb 02, 2016 10:03 am

MisterDave wrote:Hi B0b

Just seen your post. For the terrain I'm using a greyscale 8-bit image which gives you 256 possible height values. This can be scaled up for more height, but will increase the visible step size unless you do some smoothing. A 16 bit heightmap can be used but that increases memory usage.
yes, I saw that on your youtube videos.

right now I'm using 3 floats per terrain point.
height, radius of influence and error measure. that sums up to quite much ram usage, but I'm sure I can figure it out to use less.
error value could be 1 byte, height and radius could be done as shorts I think. I'm not loading the heighmap from bitmaps directly, as it is preprocessed from a bitmap and saved as an "geometry file".
i'm trying it this way because...
The approach I'm using for LoD is a fairly good for CPU usage but is likely not using the cache as efficient as I would like and also has the problem of gaps when the detail level changes. All things to fix in the future, but I need to look at a few other algorithms on vterrain.org to help inform this in addition to the WoW format.
... I'm using another algorithm for LOD. the terrain points are saved in z-order. the x and y positions are generated on the-fly from the morton code (which is the index of the point in the array). I just hope for better cache efficiency :/ the LOD is rebuild every frame (you have to scan the quadtree anyway, right?), so it's different from ROAM which uses a 2step approach. the visible vertices are then rendered as tristrips (that part was quite complicated and I'm still working on it). on the plus side - no gaps and minimal jittering
Blending the textures isn't too difficult with an RGB map (with each channel representing the intensity of each texture if you don't need too many textures) and some blurring with GIMP. It's pretty straightforward to draw a base texture then a transparent one over the top based on the RGB map. I'm using a single texture atlas so only need to bind one physical texture, however, logical textures can't be repeated per poly in the distance for low LoD polys (U/V bigger than 1), so I've had to create some doubled textures at the same size of the normal ones (and so on).
I might have a solution in rendering terrain only with opaque polys (pre-blend all the needed textures every now and then). I can post the general approach then I'm sure it could work - if anyone is interested of course
I've used the RGB also to decide where my terrain foliage and rocks go which will be in an upcoming video in the next day or so.

Rendering to texture is next on the agenda for me for DoF. I did this achieve using a modification to the original KGL a couple of years back and am currently adapting my code to work on Phenom's new API as he uses a much more modern FBO approach as far as the wrapper is concerned.

Cheers,
Dave
I'm looking forward to it :)
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has liked: 0
Been liked: 3 times

Re: Terrain texturing

Post by PH3NOM » Tue Feb 02, 2016 10:32 pm

MisterDave wrote: Rendering to texture is next on the agenda for me for DoF. I did this achieve using a modification to the original KGL a couple of years back and am currently adapting my code to work on Phenom's new API as he uses a much more modern FBO approach as far as the wrapper is concerned.

Cheers,
Dave
Hey MisterDave!

The proper use of the FBO's is demonstrated in the blur example included in the API.

However, there is another method I included in the API that is not well documented.

The traditional approach will wipe the buffers in main ram when you call glutSwapBuffers().
This means if you want the original geometry to be submitted, you need to submit it again if you plan to use the result of the render-to-texture on top of the base geometry.

On top of that, I have included a function

Code: Select all

glutCopyBufferToTexture(void *dst, GLsizei *x, GLsizei *y);
The difference is:
1.) It will render into the texture that you supply as an argument ( a FBO still needs to be bound for this to work, but the FBO buffer will not be used, rather the buffer you supply as *dst )
2.) The Vertex Buffers will not be wiped, meaning any vertex rendered-to-texture will also be submitted on the next frame.

I hope that makes sense.
When utilized correctly this will save a large amount of CPU time; as lighting, transform, clipping, etc. will only need to be done once instead of twice, per-vertex.
b0b wrote:
PH3NOM wrote:Terrain is generally very easy; the height map r,g,b values are interpreted as x,y,z, and viola, 3D terrain.
yeah, well, except it isn't if you are trying to render detailed terrain with good draw distance.
you have to implement some sort of LOD-system. nowadays on newer hardware you can do pretty much everything terrain related in shaders. on the dreamcast the SH4 is your only friend :(
there are still some papers (from around 2000-2002) around to make terrain LOD on the CPU. MisterDave here on the board seems to use the ROAM-algorithm for LOD for example (I'm trying another approach, but might also go back to ROAM if it's not good enough...)
Truth is, I have not worked on an LOD system specifically for Terrain Maps, but I have some ideas...
How much RAM are you using? Is there any way you can mip-map your "geometry file"?
Just 1/2 and 1/4 scale should provide decent results. Up to 1/8 would be ideal in most cases.
My idea is you could use a fast quadratic distance computation based on your morton code to determine which level of mip-map to load.
This would result in 0 CPU time on tessellation(pre-computed), and provide seamless stitches between each level.
Ideally, things will be (inherently) ordered as Triangle Strips for the fastest possible transfer to the PVR GPU.

Keep us updated with your progress!
-Josh
User avatar
MisterDave
DCEmu Freak
DCEmu Freak
Posts: 58
Joined: Mon Apr 08, 2013 1:16 pm
Has liked: 0
Been liked: 0

Re: Terrain texturing

Post by MisterDave » Wed Feb 03, 2016 7:47 am

Thanks PH3NOM

I did wonder about glutCopyBufferToTexture when I saw it in your library, but just assumed it was deprecated code after seeing the Blur example.

So far the rendering to a texture is working fine for me. The only problem I'm having is that disabling the depth buffer and mask isn't giving me the result I'd expect as I am drawing the texture in ortho mode, going back to perspective mode then drawing the rest of the scene after this 'on top'. Currently the texture always stays in front. In the scene the texture gets the polys beyond a distance threshold then the main frame buffer gets the textures from the camera to this threshold.
User avatar
PH3NOM
DC Developer
DC Developer
Posts: 576
Joined: Fri Jun 18, 2010 9:29 pm
Has liked: 0
Been liked: 3 times

Re: Terrain texturing

Post by PH3NOM » Fri Feb 05, 2016 1:44 pm

So you disable depth-testing completely on the 2nd pass?
I think that would produce some strange artifacts because all of the geometry on the 2nd pass will need to be Depth Sorted in software.
The way the API works, is if depth-testing is disabled, the geometry will always pass the depth-test, meaning it will simply render everything on-screen submitted in the order you submit your vertices.

A quick note, when depth testing is enabled, you can control the order of depth-testing with glDepthFunc(GLenum func). The default parameter for this is is GL_GEQUAL.

Are you submitting the render-to-texture quad as opaque or TR? Remember, the PVR automatically sorts TR polys, unless you disable auto-sort.

And what Z-value are you submitting the render-to-texture quad at?

Finally, do you submit the render-to-texture quad before all of the geometry in the 2nd pass or after all of the geometry in the 2nd pass?
User avatar
MisterDave
DCEmu Freak
DCEmu Freak
Posts: 58
Joined: Mon Apr 08, 2013 1:16 pm
Has liked: 0
Been liked: 0

Re: Terrain texturing

Post by MisterDave » Fri Feb 05, 2016 2:19 pm

Hi PH3NOM

I may have misunderstood the way that the depth mask works as my approach was to disable the mask then draw the texture in ortho mode then turn it back on then draw my scene assuming that no depth information would be written. I had been using glVertex2f to draw the texture when in ortho mode accepting the default Z value, however, I understand that in this mode the Z min/max is -1/1 . After reading around and seeing how this has been done previously my next tactic was to use a depth of 0.99... and see if that worked.

With the submission of opaque polys and transparent ones, I remember that in the old KGL you had to submit the opaque list first then transparent. With your new one I know that they can be mixed, but had just assumed that the decision on whether the poly is opaque or transparent is done based on the alpha value, but I may be wrong there.

Thanks,
Dave
User avatar
MisterDave
DCEmu Freak
DCEmu Freak
Posts: 58
Joined: Mon Apr 08, 2013 1:16 pm
Has liked: 0
Been liked: 0

Re: Terrain texturing

Post by MisterDave » Sun Feb 07, 2016 12:54 pm

Sorted it. Setting the depth of the textured quad in ortho mode to 0.999f was what I had missed.

Attached is a daytime shot with the DoF rendered in the background and again in the night time with dynamic lights.
Image Image
Post Reply