AR-Camera

AR-Camera

Requests for features in the spring code.

Moderator: Moderators

User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

AR-Camera

Post by PicassoCT »

Basically a regular Camera, with a transparent sky and the ability to place the map anyhwere in 3d space, using two coordinates.
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

--AR-Cam-API --

-- SetCameraPosDir(x,y,z,dx,dy,dz)
-- SetWorldPos(px,py,py,ox,oy,oz)

I know thats a lot of trouble, to basically turn your livingroom floor with two qr-markers into a Spring Map.

https://www.youtube.com/watch?v=dB8vmRhMwfg

Image
Senethril
Posts: 42
Joined: 04 Oct 2013, 18:20

Re: AR-Camera

Post by Senethril »

can i have this i need it
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

This is a engine change. Basically the engine needs to supply a callin that transfers the ar-devices position and direction from a device dll. This then needs to be available from widget space.

This thing is still just a idea im sorry.
I can fork stable and give it a try.
Ares
Balanced Annihilation Developer
Posts: 555
Joined: 19 Mar 2011, 13:43

Re: AR-Camera

Post by Ares »

When you project the map in 3D does it have all the units on it? Would be very cool to watch BA games in 3D
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

Some clarification:
AR = Augmented Reality.
For this you need several things:
A Displayglass - for example a cellphone- with a camera on the back.
A strong x86 pc to render the picture aka run the spring engine at a on demand max framerate basis (directionchange).

Also you do not want this without cable.


The AR-SDK running on the AR-Googles identifys Markers- which are usually QR-Code like matrices.

Image
These are used to compute the positon of the googles to the markers in space- and theire orientation. This has to be updated fast - for latency- thus the cables (also because batterys suck).

Now having these coordinates, the AR-Plugin has to hand them over to spring - which happens in a dll.

Also in this dll, the camera position of the game is updated, and a new picture is drawn, in stereoscopic way - which then traverses back up the cable to the cellphone- and is layed over the transformed camera-picture you would usually see.
Suddenly little peewees on the floor.
Last edited by PicassoCT on 18 Apr 2018, 20:19, edited 1 time in total.
Ares
Balanced Annihilation Developer
Posts: 555
Joined: 19 Mar 2011, 13:43

Re: AR-Camera

Post by Ares »

if you could watch RTS in 3D Spring would be first of its kind to do and it could attract lots of new people, really good
User avatar
OPAdolfSatan
Posts: 35
Joined: 28 Nov 2016, 10:27

Re: AR-Camera

Post by OPAdolfSatan »

Ares wrote:if you could watch RTS in 3D Spring would be first of its kind to do and it could attract lots of new people, really good


Ares why cant i pm you directly here?? yoo sh*t is block.
Ares
Balanced Annihilation Developer
Posts: 555
Joined: 19 Mar 2011, 13:43

Re: AR-Camera

Post by Ares »

Are you sure my account name was in the field and you gave the message a subject, send screenshot. I never got any pm's so maybe its broken thanks for telling me.
User avatar
OPAdolfSatan
Posts: 35
Joined: 28 Nov 2016, 10:27

Re: AR-Camera

Post by OPAdolfSatan »

Ares wrote:Are you sure my account name was in the field and you gave the message a subject, send screenshot. I never got any pm's so maybe its broken thanks for telling me.
pm me ur skype ill message you there. It wont let me send messages thro here for some reason
Ares
Balanced Annihilation Developer
Posts: 555
Joined: 19 Mar 2011, 13:43

Re: AR-Camera

Post by Ares »

Ask an admin why pm isnt working for you, or say my name and speak to the advanced fusion in the sky and I will be there for you
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

Sungod support calls may carry a additional cost of one ripped out heart per begun minute. Cost per carrier may vary.
User avatar
Forboding Angel
Evolution RTS Developer
Posts: 14673
Joined: 17 Nov 2005, 02:43

Re: AR-Camera

Post by Forboding Angel »

I FUCK IN SKY!

#deadmeme
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

https://www.wired.com/2016/06/demonical ... uter-chip/

Bumping with unconnected interesting post


This whole AR Thing is basically non-archievable without good marker tracking- googles AR-Core lib is doing a decent job for keeping spawned gimicks temporarily steady- but has no way to restore a lost out of sight plane

Image
Attachments
dissapointSetup.png
(323.9 KiB) Not downloaded yet
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

Feel free to give it a try yourself
Attachments
models.7z
(181.54 KiB) Downloaded 18 times
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

Hi, a short update on the whole AR Project.

What i did so far:

1) I forked the google Android ARCore demo projects and modified it so that it draws a recived picture over the percived Picture. It still has bugs -but its under way in a good direction.

2) I started integrating a custom AR-Cam into the current spring master. Respect to the current status of the engine- never before built so flawless.

3) The plan is to integrate lib_usb, deactivate the drawing of the sky- render the whole mess to png and send it to the android phone via usb. From the phone, we recive the poses (4x4) Matrixes of the current Camera and the anchor of the display.

4) Draw the spring on top of a table, making it a vr tabletop.

Optional) Make it true Googly AR by adding a scond camera to the AR-Cam. Also shadders to get to this:

Image

Hope i get this done before work restarts. Cheerios

Related spring repo:
https://github.com/PicassoCT/spring/tree/ardev

The Augmented App Repo is outdated- kind of threw out alot of the stuff there.
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

So what do i got so far:
A lousy little App, which can draw one picture at a time
Luasocket widget (Thanks to Pako)
A tcp client integrated into the lously little app. (should be udp later)

Image

Superslow going is super slow.
Attachments
oneStaticImageOverlayed.png
(392.85 KiB) Not downloaded yet
SquarePeg
Posts: 2
Joined: 01 May 2018, 11:41

Re: AR-Camera

Post by SquarePeg »

What version of ARCore are you using? 1.2 was released at Google I/O and It has allot of cool stuff in it.
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

Code: Select all


dependencies {
    // ARCore library
    implementation 'com.google.ar:core:1.1.0'

    // Obj - a simple Wavefront OBJ file loader
    // https://github.com/javagl/Obj
    implementation 'de.javagl:obj:0.2.1'

    implementation 'com.android.support:appcompat-v7:27.0.2'
    implementation 'com.android.support:design:27.0.2'
}

So its the old version. And yes, im building upon a pre-existing demo.
https://github.com/PicassoCT/arcore-android-sdk

And yes, its currently tcp. And full off bugs. So to all the nil-contributors out there, who just want to dive in to vomit some acid- patch is welcome.
User avatar
PicassoCT
Journeywar Developer & Mapper
Posts: 10450
Joined: 24 Jan 2006, 21:12

Re: AR-Camera

Post by PicassoCT »

The problem is thus, i get the following error message:

Code: Select all

[Codebox=txt file=Logcat] 06-20 21:02:42.995 17769-17791/com.google.ar.core.examples.java.helloar.debug D/SpringOverlayRenderer: Spring OverlayRender Update called
    Spring OverlayRender drawOverlay called
06-20 21:02:42.995 17769-17791/com.google.ar.core.examples.java.helloar.debug E/emuglGLESv2_enc: device/generic/goldfish-opengl/system/GLESv2_enc/GL2Encoder.cpp:s_glEnableVertexAttribArray:875 GL error 0x501
    Info: Invalid vertex attribute index. Wanted index: 4294967295. Max index: 16
    device/generic/goldfish-opengl/system/GLESv2_enc/GL2Encoder.cpp:s_glVertexAttribPointer:598 GL error 0x501
    Info: Invalid vertex attribute index. Wanted index: 4294967295. Max index: 16
    device/generic/goldfish-opengl/system/GLESv2_enc/GL2Encoder.cpp:s_glVertexAttribPointer:598 GL error 0x501
    Info: Invalid vertex attribute index. Wanted index: 4294967295. Max index: 16
    device/generic/goldfish-opengl/system/GLESv2_enc/GL2Encoder.cpp:s_glEnableVertexAttribArray:875 GL error 0x501
    Info: Invalid vertex attribute index. Wanted index: 4294967295. Max index: 16
06-20 21:02:42.996 17769-17791/com.google.ar.core.examples.java.helloar.debug E/SpringARActivity: Exception on the OpenGL thread
    java.lang.ArrayIndexOutOfBoundsException: remaining() < count < needed
        at android.opengl.GLES20.glDrawElements(Native Method)
        at com.google.ar.core.examples.app.common.rendering.SpringOverlayRenderer.drawOverlay(SpringOverlayRenderer.java:226)
        at com.google.ar.core.examples.app.springar.SpringARActivity.onDrawFrame(SpringARActivity.java:378)
        at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1571)
        at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1270)
06-20 21:02:42.998 1744-16525/? I/ActivityManager: Force stopping com.google.ar.core.examples.java.helloar.debug appid=10092 user=0: from pid 17834
    Killing 17769:com.google.ar.core.examples.java.helloar.debug/u0a92 (adj 0): stop com.google.ar.core.examples.java.helloar.debug
06-20 21:02:42.998 1744-1762/? W/libprocessgroup: kill(-17769, 9) failed: No such process
Pointing me to this function:

Code: Select all

public void drawOverlay(float[] cameraView, float[] cameraPerspective) {
Log.d(TAG, "Spring OverlayRender drawOverlay called");

GLES20.glUseProgram(overlayProgram);

mPositionHandle = GLES20.glGetAttribLocation(overlayProgram, "vPosition");
GLES20.glEnableVertexAttribArray(mPositionHandle);
int vsTextureCoord = GLES20.glGetAttribLocation(overlayProgram, "TexCoordIn");
GLES20.glVertexAttribPointer(mPositionHandle,
COORDS_PER_VERTEX,
GLES20.GL_FLOAT,
false,
vertexStride,
vertexBuffer);
GLES20.glVertexAttribPointer(vsTextureCoord,
COORDS_PER_TEXTURE,
GLES20.GL_FLOAT,
false,
textureStride,
textureBuffer);

GLES20.glEnableVertexAttribArray(vsTextureCoord);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[0]);
int fsTexture = GLES20.glGetUniformLocation(overlayProgram, "TexCoordOut");
GLES20.glUniform1i(fsTexture, 0);
mMVPMatrixHandle = GLES20.glGetUniformLocation(overlayProgram, "uMVPMatrix");
GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, cameraPerspective, 0);

GLES20.glDrawElements(GLES20.GL_TRIANGLES,
indices.length,
GLES20.GL_UNSIGNED_SHORT,
drawListBuffer);

GLES20.glDisableVertexAttribArray(mPositionHandle);

ShaderUtil.checkGLError(TAG, "Cleaning up after drawing planes");
}
Filled in this function:

Code: Select all

public void createOnGlThread(Context context) throws IOException {
Log.d(TAG, "Spring OverlayRender createOnGlThread called");
this.context = context;
this.tcpConnection = new Server(this);

ByteBuffer byteBuf = ByteBuffer.allocateDirect(vertices.length * 4);
byteBuf.order(ByteOrder.nativeOrder());
vertexBuffer = byteBuf.asFloatBuffer();
vertexBuffer.put(vertices);
vertexBuffer.position(0);
byteBuf = ByteBuffer.allocateDirect(texture.length * 4);

byteBuf.order(ByteOrder.nativeOrder());
textureBuffer = byteBuf.asFloatBuffer();
textureBuffer.put(texture);
textureBuffer.position(0);

ByteBuffer indexBuffer = ByteBuffer.allocateDirect(indices.length);
indexBuffer.put(indices);
indexBuffer.order(ByteOrder.nativeOrder());
drawListBuffer = indexBuffer.asShortBuffer();
drawListBuffer.position(0);

try {
vertexShader =
ShaderUtil.loadGLShader(TAG, context, GLES20.GL_VERTEX_SHADER, VERTEX_SHADER_NAME);
fragmentShader =
ShaderUtil.loadGLShader(TAG, context, GLES20.GL_FRAGMENT_SHADER, FRAGMENT_SHADER_NAME);
} catch (IOException i) {
Log.e(TAG, "GLES20::Error" + i.toString());
ShaderUtil.checkGLError(TAG, "GLES20::Error: Program parameters");
}


GLES20.glCompileShader(vertexShader);
GLES20.glCompileShader(fragmentShader);

overlayProgram = GLES20.glCreateProgram();
GLES20.glAttachShader(overlayProgram, vertexShader);
GLES20.glAttachShader(overlayProgram, fragmentShader);
GLES20.glLinkProgram(overlayProgram);
GLES20.glUseProgram(overlayProgram);

//LOad the texture


tcpConnection.Buffer.loadTexture(context,
0,
"models/springoverlayraw.png"
);
tcpConnection.Buffer.loadTexture(context,
1,
"models/springoverlayraw.png"
);

ShaderUtil.checkGLError(TAG, "Program parameters");
/* */
}
My OpenGL Foo is quite rusty. Not in the rust sort of way. So any advice is welcome.
Post Reply

Return to “Feature Requests”