Skip to content

point cloud data #147

Open
Open
point cloud data#147
Labels
exampleThis can be example implemented using the SDK

Description

@neverland-haha
neverland-haha
opened on Sep 10, 2022

What feature or requirment is this related to? Please describe
This code example relates to the feature or requirement mentioned on #<github_issue_here>

!pip install mapillary 
import mapillary as mly
mly.interface.set_access_token('MLY|')
response = mly.interface.image_from_key(id,fields = ['sfm_cluster'])
sfm_cluster_url = 
https:\\/\\/scontent-iad3-1.xx.fbcdn.net\\/m1\\/v\\/t0.40383-6\\/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a

real_url = 'https:\\scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a'

I want to get the point of cloud of each image so I can reconstruction the camera position and get better visualization, positioning, however. When I try to get the url that links to sfm_cluster, it returns to me a binary stream. After that , I write it content into a txt file,
but it becomes messy code. Is there any way to decode the messy code or api that I can get the *.ply format file?
Thank you!

Additional context
image

image

Activity

  1. neverland-haha
    added
    exampleThis can be example implemented using the SDK
    on Sep 10, 2022

  2. Rubix982 commented on Sep 10, 2022

    @Rubix982
    Rubix982
    on Sep 10, 2022
    Contributor

    Yeah, right now, there isn't any mechanism in the SDK to decode the contents from the URL that sfm_cluster returns. @cbeddow would you know of a way to do this?

  3. neverland-haha
    changed the title [-]Code Example[/-] [+]point cloud data[/+] on Sep 10, 2022

  4. neverland-haha commented on Sep 10, 2022

    @neverland-haha
    neverland-haha
    on Sep 10, 2022
    Author

    Yeah, right now, there isn't any mechanism in the SDK to decode the contents from the URL that sfm_cluster returns. @cbeddow would you know of a way to do this?

    So, it means that there is no solution to decode the contents of this url? I wonder if I could know it encoding type, then Maybe I can try to decode myself?

  5. Rubix982 commented on Sep 10, 2022

    @Rubix982
    Rubix982
    on Sep 10, 2022
    Contributor

    @neverland-haha, there is a way to decode the contents. I meant to say that this solution has not yet been integrated into the SDK.

    The sfm_cluster property returns a link that points to an encoded .ply format file.

    Can you please share the file here that you have right now? I can look into this and integrate a solution to decode the SFM as well.

  6. neverland-haha commented on Sep 10, 2022

    @neverland-haha
    neverland-haha
    on Sep 10, 2022
    Author

    @neverland-haha, there is a way to decode the contents. I meant to say that this solution has not yet been integrated into the SDK.

    The sfm_cluster property returns a link that points to an encoded .ply format file.

    Can you please share the file here that you have right now? I can look into this and integrate a solution to decode the SFM as well.

    This this the url tha sfm_cluster returns.
    https:\\scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a

  7. Rubix982 commented on Sep 10, 2022

    @Rubix982
    Rubix982
    on Sep 10, 2022
    Contributor

    @neverland-haha, I was able to write a quick python script for this.

    import requests
import json
import zlib

def main():
    # change URL here as per need
    sfm_content_url = 'https://scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a'

    # decompress sfm content, and return dict
    print(decompress_sfm_content(sfm_content_url))

def decompress_sfm_content(sfm_content_url: str) -> dict:
    # make get request with python and get content

    response = requests.get(sfm_content_url)
    content = response.content

    # decompress zlib content
    return json.loads(zlib.decompress(content).decode('utf-8'))

if __name__ == '__main__':
    main()

    I'll be adding this function soon in the SDK. Please let me know if this resolves your concern.

    And thanks for opening this issue! I learned something new today.

  8. neverland-haha commented on Sep 10, 2022

    @neverland-haha
    neverland-haha
    on Sep 10, 2022
    Author

    @neverland-haha, I was able to write a quick python script for this.

    import requests
import json
import zlib

def main():
    # change URL here as per need
    sfm_content_url = 'https://scontent-iad3-1.xx.fbcdn.net/m1/v/t0.40383-6/An8Lx-3QlYDKv2YxzpOgOEJeMdx1v4mm8Xvczm4oxncKAIlA_cmDpxwXsy65cnsTUmM6mgq9dpooROG658tIhEyOIrA8uHcqeFK5gB9TMikR9788Ku_awoEbtFHGJg2X2jPKjy0padhn0w?ccb=10-5&oh=00_AT_Tar_zzekzP0jDvbYAeHCNxU7GITO5m5HtEYy95wwm2w&oe=632180A9&_nc_sid=0f319a'

    # decompress sfm content, and return dict
    print(decompress_sfm_content(sfm_content_url))

def decompress_sfm_content(sfm_content_url: str) -> dict:
    # make get request with python and get content

    response = requests.get(sfm_content_url)
    content = response.content

    # decompress zlib content
    return json.loads(zlib.decompress(content).decode('utf-8'))

if __name__ == '__main__':
    main()

    I'll be adding this function soon in the SDK. Please let me know if this resolves your concern.

    And thanks for opening this issue! I learned something new today.

    It does solve my problem, Thank you very much!

  9. neverland-haha
    closed this as completedon Sep 11, 2022

  10. Rubix982 commented on Sep 11, 2022

    @Rubix982
    Rubix982
    on Sep 11, 2022
    Contributor

    @neverland-haha, I'll close when I finally merge the PR for this function in the SDK. Reopening for now.

  11. Rubix982
    reopened this on Sep 11, 2022

  12. neverland-haha commented on Sep 11, 2022

    @neverland-haha
    neverland-haha
    on Sep 11, 2022
    Author

    @neverland-haha, I'll close when I finally merge the PR for this function in the SDK. Reopening for now.

    import mapillary as mly
import requests 
import requests
import json
import zlib

mly.interface.set_access_token('MLY|5175886149161658|4099af83a040b4c64a3891bdcc8314cf')
data = mly.interface.image_from_key('1239407999827329',fields = ['sfm_cluster'])
result = eval(data)
url = result['features']['properties']['sfm_cluster']['url'].replace('\\','')
response = requests.get(url,stream=True,timeout=10)
info = json.loads(zlib.decompress(response.content).decode('utf-8'))
str_info = str(info)
length = len(str_info)
real_info = str_info[1:length-1]
dict_info = eval(real_info)
print(dict_info)

    The dict_info contains too much infomation, but I want to retrive the earing angle and the pitch angle of the camera from the point cloud data. How can I know that point that I want?The mly.interface.image_from_key function returns me the id of point of cloud. But I can't find it in dict_info.

  13. Rubix982 commented on Sep 12, 2022

    @Rubix982
    Rubix982
    on Sep 12, 2022
    Contributor

    Hi, @neverland-haha! I haven't yet had the time to explore this, but some intuition that I can help with maybe.

    After storing the data as a JSON, I found there are 4 keys in the dictionary.

    image

    The shots and points are the most heavy data keys.

    I'm not sure myself what to do for the earing angle or the pitch angle in this case (I still lack some more advanced GIS knowledge), but I found this field that may (?) help you,

    image

    @cbeddow what would you suggest to do here in this case?

  14. cbeddow commented on Sep 12, 2022

    @cbeddow
    cbeddow
    on Sep 12, 2022
    Contributor

    @Rubix982 @neverland-haha in the latest JSON example above, if you need the camera bearing angle, you will work with the rotation of the image: https://opensfm.org/docs/cam_coord_system.html

    It should be the Z-Axis is telling you which angle the camera is facing. I am not sure if that is a 0 to 360 value though. The full data there you see is designed to work with OpenSfM library in Python so I suggest to go from Mapillary Python SDK to OpenSfM to work with the results you get from the JSON: https://opensfm.org/docs/dataset.html#reconstruction-file-format

  15. neverland-haha commented on Sep 13, 2022

    @neverland-haha
    neverland-haha
    on Sep 13, 2022
    Author

    @Rubix982 @neverland-haha in the latest JSON example above, if you need the camera bearing angle, you will work with the rotation of the image: https://opensfm.org/docs/cam_coord_system.html

    It should be the Z-Axis is telling you which angle the camera is facing. I am not sure if that is a 0 to 360 value though. The full data there you see is designed to work with OpenSfM library in Python so I suggest to go from Mapillary Python SDK to OpenSfM to work with the results you get from the JSON: https://opensfm.org/docs/dataset.html#reconstruction-file-format

    Thanks, I' ll try and give you the answer. But As the doc says: Y-Axis is telling the angle the camera is up or down, however, when I change the units from rad to degree,some angle even greater than 90°. I'll check the process again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    exampleThis can be example implemented using the SDK

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions