Unable to download files using Data Management API+OSS signed S3 url

[hardik_malik9ZCEX]

I am developing a web application wherein I am downloading ACC files via Data Management API and OSS signed URLs stops around 10MB and doesn't complete. The signed S3 URL is generated successfully, but the download stalls.

APIs Used:

1. Data Management API v1 - Get file versions and storage location

1. OSS API v2 - Get signed S3 download URL

1. Direct S3 download - Download file from signed URL.

Code Implementation:

1) Get storage location (Data Management API):

versions_url = f"https://developer.api.autodesk.com/data/v1/projects/{project_id}/items/{item_id}/versions"
params = {"page[limit]": 1, "sort": "-versionNumber"}
headers = {"Authorization": f"Bearer {aps_token}"}

response = requests.get(versions_url, headers=headers, params=params)
storage_location = response.json()["data"][0]["relationships"]["storage"]["data"]["id"]
# Returns: "urn:adsk.objects:os.object:wip.dm.prod/filename.rvt"

2) Get signed S3 URL (OSS API):

bucket, object_key = storage_location.replace("urn:adsk.objects:os.object:", "").split("/", 1)
signed_url_endpoint = f"https://developer.api.autodesk.com/oss/v2/buckets/{bucket}/objects/{object_key}/signeds3download"

signed_response = requests.get(signed_url_endpoint, headers=headers)
signed_s3_url = signed_response.json()["url"]
# Returns: "https://s3.amazonaws.com/com.autodesk.oss-persistent/...signed..."

3) Download file (Direct S3):

response = requests.get(signed_s3_url, stream=True, timeout=(60, 1800))
response.raise_for_status()

with open(temp_file, "wb") as f:
    for chunk in response.iter_content():  # Default chunk size
        if chunk:
            f.write(chunk)

I have also tried breaking it into smaller chunks

4) Download URL Code:

def _get_file_download_url(self, file_id, project_id, aps_token):
        """
        Get the download URL for a file from ACC using Data Management API
        
        Args:
            file_id: The ACC file ID (item ID)
            project_id: The project ID containing the file
            aps_token: Authentication token
            
        Returns:
            tuple: (download_url, storage_location) or (None, None) if failed
        """
        try:
            # Clean the project and item IDs
            # Ensure project ID has 'b.' prefix
            clean_project_id = project_id if project_id.startswith('b.') else f"b.{project_id}"
            
            # URL encode the item ID (should already be a URN)
            encoded_item_id = requests.utils.quote(file_id, safe='')
            
            # Get versions to find the storage location
            versions_url = f"https://developer.api.autodesk.com/data/v1/projects/{clean_project_id}/items/{encoded_item_id}/versions"
            params = {
                "page[limit]": 1,
                "sort": "-versionNumber"
            }
            headers = {"Authorization": f"Bearer {aps_token}"}
            
            print(f"[BACKUP] Getting file versions from: {versions_url}")
            print(f"[BACKUP] Project ID: {clean_project_id}")
            print(f"[BACKUP] Item ID: {encoded_item_id}")
            
            response = requests.get(versions_url, headers=headers, params=params)
            
            if response.status_code != 200:
                print(f"[BACKUP] Failed to get versions: {response.status_code}")
                print(f"[BACKUP] Response: {response.text}")
                return None, None
            
            data = response.json()
            print(f"[BACKUP] Response data: {json.dumps(data, indent=2)}")
            versions = data.get("data", [])
            
            if not versions:
                print("[BACKUP] No versions found")
                print(f"[BACKUP] Full response: {data}")
                return None, None
            
            # Get storage location from the latest version
            latest_version = versions[0]
            print(f"[BACKUP] Latest version structure: {json.dumps(latest_version.get('relationships', {}), indent=2)}")
            storage_location = latest_version.get("relationships", {}).get("storage", {}).get("data", {}).get("id")
            
            if not storage_location:
                print("[BACKUP] No storage location found")
                print(f"[BACKUP] Version data available: {latest_version.keys()}")
                return None, None
            
            print(f"[BACKUP] Found storage location: {storage_location}")
            
            # Extract bucket and object key from storage location
            # Storage location format: urn:adsk.objects:os.object:bucket/object_key
            storage_parts = storage_location.replace("urn:adsk.objects:os.object:", "").split("/", 1)
            if len(storage_parts) != 2:
                print(f"[BACKUP] Invalid storage location format: {storage_location}")
                return None, None
            
            bucket_key = storage_parts[0]
            object_key = storage_parts[1]
            
            # URL encode the object key for the API call
            encoded_object_key = requests.utils.quote(object_key, safe='')
            
            # Get signed download URL from OSS (Object Storage Service)
            # Use the signeds3download endpoint to get a temporary download URL
            signed_url_endpoint = f"https://developer.api.autodesk.com/oss/v2/buckets/{bucket_key}/objects/{encoded_object_key}"
            
            print(f"[BACKUP] OSS endpoint: {signed_url_endpoint}")
            
            # First, let's try to get a signed download URL
            headers = {"Authorization": f"Bearer {aps_token}"}
            
            # Try getting the object with a signed URL request
            try:
                signed_response = requests.get(f"{signed_url_endpoint}/signeds3download", headers=headers)
                if signed_response.status_code == 200:
                    signed_data = signed_response.json()
                    if 'url' in signed_data:
                        print(f"[BACKUP] Got signed download URL")
                        return signed_data['url'], storage_location
            except Exception as e:
                print(f"[BACKUP] Could not get signed URL: {e}")
            
            # Fallback to direct download URL
            download_url = signed_url_endpoint
            print(f"[BACKUP] Using direct download URL: {download_url}")
            return download_url, storage_location
            
        except Exception as e:
            print(f"[BACKUP] Error getting download URL: {e}")
            return None, None
    

Problems:

• Signed URL is generated successfully

• Download starts and progresses

• Stops around 10MB (varies)

• No error thrown; connection appears to hang

• File size: ~78MB

• Timeout set to 30 minutes (1800s)

What I've Tried:

• Increased timeout from 5 min to 30 min

• Tried different chunk sizes (1MB, 512KB, default)

• Verified network connectivity (can access S3 URL directly in browser)

• Checked token scopes (data:read, bucket:read)

 Questions:

1. Are there known issues with large file downloads (>50MB) from signed S3 URLs?

1. Should I use a different approach for large files?

1. Are there recommended chunk sizes or timeout values?

1. Could network proxies/firewalls interfere with long-running S3 downloads?