Merge pull request #4 from mattintech/bug/largeupload
Add QR code scanning for server configuration
This commit is contained in:
GitHub
66
PROJECT.md
Normal file
66
PROJECT.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# SimpleLogUpload Project Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
SimpleLogUpload is an Android application designed to upload log files to a server. It provides both standard and chunked upload capabilities to handle files of various sizes across different network conditions.
|
||||
|
||||
## Documentation Structure
|
||||
|
||||
This project follows specific documentation conventions:
|
||||
|
||||
1. **Main documentation**:
|
||||
- `README.md` - General project information and setup instructions
|
||||
- `PROJECT.md` (this file) - Project structure and documentation guidelines
|
||||
|
||||
2. **Feature documentation**:
|
||||
- Each feature must have its own dedicated Markdown file in the `./docs` folder
|
||||
- Feature documentation should thoroughly explain the implementation, usage, and any special considerations
|
||||
|
||||
3. **Task tracking**:
|
||||
- `TASK.md` - Contains ongoing tasks and development roadmap
|
||||
|
||||
## Documentation Requirements
|
||||
|
||||
### Feature Documentation Rule
|
||||
|
||||
**IMPORTANT**: For each new feature added to the project, a corresponding Markdown file must be created in the `./docs` folder. This file should:
|
||||
|
||||
1. Have a clear, descriptive filename (e.g., `FEATURE_NAME.md`)
|
||||
2. Contain comprehensive documentation including:
|
||||
- Feature overview and purpose
|
||||
- Technical implementation details
|
||||
- API descriptions (if applicable)
|
||||
- Usage examples
|
||||
- Configuration options
|
||||
- Testing considerations
|
||||
|
||||
### Current Feature Documentation
|
||||
|
||||
The current feature documentation includes:
|
||||
|
||||
- `docs/CHUNKED_UPLOAD.md` - Details on the chunked upload implementation
|
||||
- `docs/LARGE_UPLOAD.md` - Analysis and improvements for large file uploads
|
||||
- `docs/RELEASE_PROCESS.md` - Guidelines for the release process
|
||||
|
||||
## Maintaining Documentation
|
||||
|
||||
When modifying or extending existing features, ensure the corresponding documentation is updated to reflect the changes. Keep all documentation accurate and up-to-date with the current implementation.
|
||||
|
||||
## Adding New Documentation
|
||||
|
||||
To add documentation for a new feature:
|
||||
|
||||
1. Create a new Markdown file in the `./docs` folder
|
||||
2. Follow the naming convention `FEATURE_NAME.md`
|
||||
3. Use clear sections and subsections for easy navigation
|
||||
4. Include code examples where appropriate
|
||||
5. Add links to the new documentation in other relevant documents
|
||||
|
||||
## Documentation Standards
|
||||
|
||||
All documentation should:
|
||||
- Be clear and concise
|
||||
- Include code examples where applicable
|
||||
- Explain the "why" as well as the "how"
|
||||
- Be kept up-to-date with code changes
|
||||
- Follow a consistent format and style
|
||||
@@ -48,6 +48,15 @@ Note - this requires that you leverage SimpleFileUpload-Server
|
||||
- Upload history filtering and search
|
||||
- QR code generation for quick sharing
|
||||
|
||||
## Documentation
|
||||
|
||||
Detailed documentation about specific features and processes can be found in the `docs/` directory:
|
||||
|
||||
- [Project Documentation](PROJECT.md) - Project structure and documentation guidelines
|
||||
- [Chunked Upload](docs/CHUNKED_UPLOAD.md) - Implementation of chunked uploads for large files
|
||||
- [Large File Upload Analysis](docs/LARGE_UPLOAD.md) - Analysis and improvements for large file uploads
|
||||
- [Release Process](docs/RELEASE_PROCESS.md) - Detailed release process guidelines
|
||||
|
||||
## Development and Releases
|
||||
|
||||
### Automated Releases with GitHub Actions
|
||||
|
||||
10
TASK.md
10
TASK.md
@@ -84,3 +84,13 @@
|
||||
- [x] Multi-Dumpstate upload
|
||||
- [ ] Dumpstate cleanup after successful upload
|
||||
- [ ] Provide steps of how to collect a dumpstate and download the server
|
||||
|
||||
## 🔄 Chunked Upload Implementation
|
||||
- [x] Implement basic chunked upload system for large files (>100MB)
|
||||
- [x] Add network type detection and adaptive settings for WiFi vs. Mobile
|
||||
- [x] Implement resume capability for interrupted uploads
|
||||
- [x] Add parallel chunk uploading for WiFi connections
|
||||
- [x] Create network state monitoring to handle connection changes
|
||||
- [x] Implement progressive backoff and retry strategy
|
||||
- [x] Add enhanced progress tracking for chunks
|
||||
- [x] Test on both WiFi and 5G with various file sizes
|
||||
|
||||
@@ -56,6 +56,9 @@ dependencies {
|
||||
implementation 'com.android.support:support-annotations:28.0.0'
|
||||
implementation 'androidx.palette:palette:1.0.0'
|
||||
|
||||
// QR Code scanning
|
||||
implementation 'com.journeyapps:zxing-android-embedded:4.3.0'
|
||||
|
||||
// Room
|
||||
def room_version = "2.6.1"
|
||||
implementation "androidx.room:room-runtime:$room_version"
|
||||
|
||||
@@ -3,11 +3,13 @@
|
||||
package="com.mattintech.simplelogupload">
|
||||
|
||||
<uses-permission android:name="android.permission.INTERNET" />
|
||||
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
|
||||
<uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE" />
|
||||
<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
||||
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_DATA_SYNC" />
|
||||
<uses-permission android:name="android.permission.CAMERA" />
|
||||
|
||||
<application
|
||||
android:allowBackup="false"
|
||||
|
||||
@@ -8,11 +8,25 @@ public class NetworkConstants {
|
||||
// via PreferencesUtil to allow for customization
|
||||
public static final String SERVER_URL_BASE = "/upload";
|
||||
|
||||
// Chunked upload endpoints (relative to server base URL)
|
||||
public static final String CHUNKED_UPLOAD_START = "/upload/start";
|
||||
public static final String CHUNKED_UPLOAD_CHUNK = "/upload/chunk";
|
||||
public static final String CHUNKED_UPLOAD_COMPLETE = "/upload/complete";
|
||||
|
||||
// Upload parameters
|
||||
public static final String PARAM_FILE = "file";
|
||||
public static final String PARAM_MAX_DOWNLOADS = "max_downloads";
|
||||
public static final String PARAM_EXPIRY_HOURS = "expiry_hours";
|
||||
|
||||
// Chunked upload parameters
|
||||
public static final String PARAM_UPLOAD_ID = "upload_id";
|
||||
public static final String PARAM_CHUNK_INDEX = "chunk_index";
|
||||
public static final String PARAM_TOTAL_CHUNKS = "total_chunks";
|
||||
public static final String PARAM_CHUNK = "chunk";
|
||||
public static final String PARAM_FILENAME = "filename";
|
||||
public static final String PARAM_FILESIZE = "filesize";
|
||||
public static final String PARAM_CHUNK_SIZE = "chunk_size";
|
||||
|
||||
// Authentication
|
||||
public static final String CLIENT_KEY_HEADER = "X-Client-Key";
|
||||
|
||||
|
||||
@@ -11,6 +11,8 @@ import com.mattintech.simplelogupload.db.UploadHistoryRepository;
|
||||
import com.mattintech.simplelogupload.model.UploadHistory;
|
||||
import com.mattintech.simplelogupload.model.UploadResult;
|
||||
import com.mattintech.simplelogupload.service.FileUploadForegroundService;
|
||||
import com.mattintech.simplelogupload.upload.ChunkedUploadManager;
|
||||
import com.mattintech.simplelogupload.upload.UploadStrategySelector;
|
||||
import com.mattintech.simplelogupload.util.PreferencesUtil;
|
||||
|
||||
import java.io.File;
|
||||
@@ -66,7 +68,14 @@ public class UploadController {
|
||||
serviceIntent.putExtra(FileUploadForegroundService.EXTRA_MAX_DOWNLOADS, NetworkConstants.DEFAULT_MAX_DOWNLOADS);
|
||||
serviceIntent.putExtra(FileUploadForegroundService.EXTRA_EXPIRY_HOURS, NetworkConstants.DEFAULT_EXPIRY_HOURS);
|
||||
|
||||
context.startService(serviceIntent);
|
||||
// On Android 12+ (API 31+), we need to use startForegroundService
|
||||
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.S) {
|
||||
Log.d(TAG, "Starting foreground service (Android 12+)");
|
||||
context.startForegroundService(serviceIntent);
|
||||
} else {
|
||||
Log.d(TAG, "Starting service (pre-Android 12)");
|
||||
context.startService(serviceIntent);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -86,6 +95,106 @@ public class UploadController {
|
||||
return future;
|
||||
}
|
||||
|
||||
// Determine whether to use chunked upload or standard upload
|
||||
boolean useChunkedUpload = UploadStrategySelector.shouldUseChunkedUpload(context, file);
|
||||
String networkType = UploadStrategySelector.getNetworkTypeName(context);
|
||||
|
||||
Log.d(TAG, "Uploading file: " + file.getName() + ", Size: " + file.length() +
|
||||
", Network: " + networkType + ", Chunked: " + useChunkedUpload);
|
||||
|
||||
if (useChunkedUpload) {
|
||||
// Use chunked upload for large files
|
||||
return performChunkedUpload(file, maxDownloads, expiryHours);
|
||||
} else {
|
||||
// Use standard upload for smaller files
|
||||
return performStandardUpload(file, maxDownloads, expiryHours);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Perform a chunked upload for large files
|
||||
*
|
||||
* @param file The file to upload
|
||||
* @param maxDownloads Maximum number of downloads
|
||||
* @param expiryHours Expiry time in hours
|
||||
* @return CompletableFuture containing the upload result
|
||||
*/
|
||||
private CompletableFuture<UploadResult> performChunkedUpload(
|
||||
File file, int maxDownloads, int expiryHours) {
|
||||
CompletableFuture<UploadResult> future = new CompletableFuture<>();
|
||||
|
||||
// Create chunked upload manager
|
||||
ChunkedUploadManager uploadManager = new ChunkedUploadManager(
|
||||
context, file, maxDownloads, expiryHours);
|
||||
|
||||
// Set progress listener if needed
|
||||
uploadManager.setProgressListener(new ChunkedUploadManager.UploadProgressListener() {
|
||||
@Override
|
||||
public void onProgress(long bytesUploaded, long totalBytes, int percentComplete) {
|
||||
// This could be used to update UI or notify a service
|
||||
Log.d(TAG, "Chunked upload progress: " + percentComplete + "%");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onChunkComplete(int chunkIndex, int totalChunks) {
|
||||
Log.d(TAG, "Chunk " + chunkIndex + "/" + totalChunks + " complete");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onUploadComplete(UploadResult result) {
|
||||
Log.d(TAG, "Chunked upload complete: " + result);
|
||||
|
||||
// Save to history
|
||||
if (result.isSuccess()) {
|
||||
saveToHistory(result);
|
||||
}
|
||||
|
||||
future.complete(result);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onError(String error) {
|
||||
Log.e(TAG, "Chunked upload error: " + error);
|
||||
future.complete(new UploadResult(error));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onPaused() {
|
||||
Log.d(TAG, "Chunked upload paused");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onResumed() {
|
||||
Log.d(TAG, "Chunked upload resumed");
|
||||
}
|
||||
});
|
||||
|
||||
// Start the upload
|
||||
uploadManager.startUpload()
|
||||
.thenAccept(result -> {
|
||||
// This is handled by the listener, but we need to catch any exceptions
|
||||
})
|
||||
.exceptionally(ex -> {
|
||||
Log.e(TAG, "Exception during chunked upload", ex);
|
||||
future.complete(new UploadResult("Exception during upload: " + ex.getMessage()));
|
||||
return null;
|
||||
});
|
||||
|
||||
return future;
|
||||
}
|
||||
|
||||
/**
|
||||
* Perform a standard (non-chunked) upload for smaller files
|
||||
*
|
||||
* @param file The file to upload
|
||||
* @param maxDownloads Maximum number of downloads
|
||||
* @param expiryHours Expiry time in hours
|
||||
* @return CompletableFuture containing the upload result
|
||||
*/
|
||||
private CompletableFuture<UploadResult> performStandardUpload(
|
||||
File file, int maxDownloads, int expiryHours) {
|
||||
CompletableFuture<UploadResult> future = new CompletableFuture<>();
|
||||
|
||||
// Create OkHttpClient with timeout configuration
|
||||
OkHttpClient client = new OkHttpClient.Builder()
|
||||
.connectTimeout(NetworkConstants.CONNECTION_TIMEOUT, TimeUnit.SECONDS)
|
||||
@@ -94,7 +203,7 @@ public class UploadController {
|
||||
.retryOnConnectionFailure(true)
|
||||
.build();
|
||||
|
||||
Log.d(TAG, "Uploading file: " + file.getName() + ", Size: " + file.length());
|
||||
Log.d(TAG, "Using standard upload for file: " + file.getName());
|
||||
|
||||
RequestBody fileBody = RequestBody.create(file, MediaType.parse("application/octet-stream"));
|
||||
MultipartBody requestBody = new MultipartBody.Builder()
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
package com.mattintech.simplelogupload.model;
|
||||
|
||||
import org.json.JSONException;
|
||||
import org.json.JSONObject;
|
||||
|
||||
/**
|
||||
* Model class for QR code server configuration
|
||||
*/
|
||||
public class QrServerConfig {
|
||||
private final String server;
|
||||
private final int port;
|
||||
private final String key;
|
||||
|
||||
private QrServerConfig(String server, int port, String key) {
|
||||
this.server = server;
|
||||
this.port = port;
|
||||
this.key = key;
|
||||
}
|
||||
|
||||
public String getServer() {
|
||||
return server;
|
||||
}
|
||||
|
||||
public int getPort() {
|
||||
return port;
|
||||
}
|
||||
|
||||
public String getKey() {
|
||||
return key;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse QR code JSON data
|
||||
*
|
||||
* @param qrData The QR code string data
|
||||
* @return QrServerConfig object
|
||||
* @throws QrConfigException if parsing fails or data is invalid
|
||||
*/
|
||||
public static QrServerConfig fromQrData(String qrData) throws QrConfigException {
|
||||
if (qrData == null || qrData.trim().isEmpty()) {
|
||||
throw new QrConfigException("QR code data is empty");
|
||||
}
|
||||
|
||||
try {
|
||||
JSONObject json = new JSONObject(qrData);
|
||||
|
||||
// Extract fields
|
||||
if (!json.has("server")) {
|
||||
throw new QrConfigException("Missing 'server' field in QR code");
|
||||
}
|
||||
if (!json.has("port")) {
|
||||
throw new QrConfigException("Missing 'port' field in QR code");
|
||||
}
|
||||
if (!json.has("key")) {
|
||||
throw new QrConfigException("Missing 'key' field in QR code");
|
||||
}
|
||||
|
||||
String server = json.getString("server");
|
||||
int port = json.getInt("port");
|
||||
String key = json.getString("key");
|
||||
|
||||
// Validate
|
||||
if (server.trim().isEmpty()) {
|
||||
throw new QrConfigException("Server address cannot be empty");
|
||||
}
|
||||
if (port <= 0 || port > 65535) {
|
||||
throw new QrConfigException("Port must be between 1 and 65535");
|
||||
}
|
||||
if (key.trim().isEmpty()) {
|
||||
throw new QrConfigException("Client key cannot be empty");
|
||||
}
|
||||
|
||||
return new QrServerConfig(server.trim(), port, key.trim());
|
||||
|
||||
} catch (JSONException e) {
|
||||
throw new QrConfigException("Invalid QR code format: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Custom exception for QR configuration errors
|
||||
*/
|
||||
public static class QrConfigException extends Exception {
|
||||
public QrConfigException(String message) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -23,6 +23,7 @@ import com.mattintech.simplelogupload.constant.NetworkConstants;
|
||||
import com.mattintech.simplelogupload.controller.FileController;
|
||||
import com.mattintech.simplelogupload.controller.UploadController;
|
||||
import com.mattintech.simplelogupload.model.UploadResult;
|
||||
import com.mattintech.simplelogupload.upload.UploadStrategySelector;
|
||||
import com.mattintech.simplelogupload.view.MainActivity;
|
||||
|
||||
import java.io.File;
|
||||
@@ -92,9 +93,17 @@ public class FileUploadForegroundService extends Service {
|
||||
}
|
||||
|
||||
// Create a notification and run the service in the foreground
|
||||
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
|
||||
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
|
||||
// For Android 14+ (API 34+)
|
||||
startForeground(NOTIFICATION_ID, createInitialNotification(), ServiceInfo.FOREGROUND_SERVICE_TYPE_DATA_SYNC);
|
||||
try {
|
||||
Log.d(TAG, "Starting foreground service with DATA_SYNC type");
|
||||
int dataSync = 1; // ServiceInfo.FOREGROUND_SERVICE_TYPE_DATA_SYNC value is 1
|
||||
startForeground(NOTIFICATION_ID, createInitialNotification(), dataSync);
|
||||
} catch (Exception e) {
|
||||
Log.e(TAG, "Error starting foreground service with type", e);
|
||||
stopSelf();
|
||||
return START_NOT_STICKY;
|
||||
}
|
||||
} else {
|
||||
// For Android 13 and below
|
||||
startForeground(NOTIFICATION_ID, createInitialNotification());
|
||||
@@ -153,7 +162,17 @@ public class FileUploadForegroundService extends Service {
|
||||
// Update notification to show uploading
|
||||
updateNotification("Uploading " + file.getName() + "...");
|
||||
|
||||
// Upload the file
|
||||
// Log network information
|
||||
String networkType = UploadStrategySelector.getNetworkTypeName(this);
|
||||
boolean isWifi = UploadStrategySelector.isWifiConnection(this);
|
||||
boolean useChunked = UploadStrategySelector.shouldUseChunkedUpload(this, file);
|
||||
|
||||
Log.d(TAG, "Starting upload - Network: " + networkType + ", Chunked: " + useChunked);
|
||||
|
||||
// Update notification with network info
|
||||
updateNotification("Uploading on " + networkType +
|
||||
(useChunked ? " (chunked)" : "") + "...");
|
||||
|
||||
uploadController.uploadFile(file, maxDownloads, expiryHours)
|
||||
.thenAccept(result -> {
|
||||
if (result.isSuccess()) {
|
||||
@@ -251,4 +270,4 @@ public class FileUploadForegroundService extends Service {
|
||||
sendBroadcast(intent);
|
||||
LocalBroadcastManager.getInstance(this).sendBroadcast(intent);
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,95 @@
|
||||
package com.mattintech.simplelogupload.upload;
|
||||
|
||||
import android.content.Context;
|
||||
import android.net.ConnectivityManager;
|
||||
import android.net.NetworkInfo;
|
||||
import android.util.Log;
|
||||
|
||||
import com.mattintech.simplelogupload.constant.AppConstants;
|
||||
|
||||
import java.io.File;
|
||||
|
||||
/**
|
||||
* Helper class to determine the best upload strategy based on file size and network conditions
|
||||
*/
|
||||
public class UploadStrategySelector {
|
||||
private static final String TAG = AppConstants.APP_TAG + "UploadStrategy";
|
||||
|
||||
// Threshold for using chunked uploads
|
||||
private static final long CHUNKED_UPLOAD_THRESHOLD_WIFI = 50 * 1024 * 1024; // 50MB for WiFi
|
||||
private static final long CHUNKED_UPLOAD_THRESHOLD_MOBILE = 10 * 1024 * 1024; // 10MB for mobile
|
||||
|
||||
/**
|
||||
* Check if chunked uploads should be used for the given file
|
||||
*
|
||||
* @param context Application context
|
||||
* @param file The file to upload
|
||||
* @return True if chunked upload should be used, false otherwise
|
||||
*/
|
||||
public static boolean shouldUseChunkedUpload(Context context, File file) {
|
||||
if (file == null) {
|
||||
return false;
|
||||
}
|
||||
|
||||
long fileSize = file.length();
|
||||
boolean isWifiConnection = isWifiConnection(context);
|
||||
long threshold = isWifiConnection ? CHUNKED_UPLOAD_THRESHOLD_WIFI : CHUNKED_UPLOAD_THRESHOLD_MOBILE;
|
||||
|
||||
boolean useChunked = fileSize > threshold;
|
||||
Log.d(TAG, "File size: " + fileSize + " bytes, Connection: " + (isWifiConnection ? "WiFi" : "Mobile") +
|
||||
", Using chunked upload: " + useChunked);
|
||||
|
||||
return useChunked;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the device is currently connected to WiFi
|
||||
*
|
||||
* @param context Application context
|
||||
* @return True if connected to WiFi, false otherwise
|
||||
*/
|
||||
public static boolean isWifiConnection(Context context) {
|
||||
ConnectivityManager cm = (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE);
|
||||
NetworkInfo activeNetwork = cm.getActiveNetworkInfo();
|
||||
return activeNetwork != null && activeNetwork.getType() == ConnectivityManager.TYPE_WIFI;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the network type name for logging
|
||||
*
|
||||
* @param context Application context
|
||||
* @return String representing the network type ("WiFi", "Mobile", or "Unknown")
|
||||
*/
|
||||
public static String getNetworkTypeName(Context context) {
|
||||
ConnectivityManager cm = (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE);
|
||||
NetworkInfo info = cm.getActiveNetworkInfo();
|
||||
|
||||
if (info == null || !info.isConnected()) {
|
||||
return "Unknown";
|
||||
}
|
||||
|
||||
if (info.getType() == ConnectivityManager.TYPE_WIFI) {
|
||||
return "WiFi";
|
||||
}
|
||||
|
||||
if (info.getType() == ConnectivityManager.TYPE_MOBILE) {
|
||||
String subType = "";
|
||||
switch (info.getSubtype()) {
|
||||
case 13: // LTE
|
||||
subType = "LTE";
|
||||
break;
|
||||
case 19: // 5G
|
||||
subType = "5G";
|
||||
break;
|
||||
case 20: // 5G+
|
||||
subType = "5G+";
|
||||
break;
|
||||
default:
|
||||
subType = info.getSubtypeName();
|
||||
}
|
||||
return "Mobile (" + subType + ")";
|
||||
}
|
||||
|
||||
return "Unknown";
|
||||
}
|
||||
}
|
||||
@@ -24,7 +24,8 @@ public class PermissionUtil {
|
||||
|
||||
public static final int REQUEST_MANAGE_EXTERNAL_STORAGE = 101;
|
||||
public static final int REQUEST_NOTIFICATION_PERMISSION = 102;
|
||||
|
||||
public static final int REQUEST_CAMERA_PERMISSION = 103;
|
||||
|
||||
/**
|
||||
* Checks if the app has notification permission
|
||||
*
|
||||
@@ -93,4 +94,31 @@ public class PermissionUtil {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if the app has camera permission
|
||||
*
|
||||
* @param context The context
|
||||
* @return true if the app has camera permission, false otherwise
|
||||
*/
|
||||
public static boolean hasCameraPermission(Context context) {
|
||||
return ContextCompat.checkSelfPermission(context, Manifest.permission.CAMERA)
|
||||
== PackageManager.PERMISSION_GRANTED;
|
||||
}
|
||||
|
||||
/**
|
||||
* Requests camera permission
|
||||
*
|
||||
* @param activity The activity
|
||||
*/
|
||||
public static void requestCameraPermission(Activity activity) {
|
||||
if (!hasCameraPermission(activity)) {
|
||||
Log.d(TAG, "Requesting camera permission");
|
||||
ActivityCompat.requestPermissions(
|
||||
activity,
|
||||
new String[]{Manifest.permission.CAMERA},
|
||||
REQUEST_CAMERA_PERMISSION
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
package com.mattintech.simplelogupload.util;
|
||||
|
||||
import com.journeyapps.barcodescanner.ScanOptions;
|
||||
import com.mattintech.simplelogupload.model.QrServerConfig;
|
||||
|
||||
/**
|
||||
* Utility class for QR code scanning
|
||||
*/
|
||||
public class QrScannerUtil {
|
||||
|
||||
/**
|
||||
* Callback interface for QR scan results
|
||||
*/
|
||||
public interface QrScanCallback {
|
||||
void onQrScanned(QrServerConfig config);
|
||||
void onQrScanCancelled();
|
||||
void onQrScanError(String errorMessage);
|
||||
}
|
||||
|
||||
/**
|
||||
* Create scan options for QR code scanning
|
||||
*
|
||||
* @return ScanOptions configured for QR codes
|
||||
*/
|
||||
public static ScanOptions createScanOptions() {
|
||||
ScanOptions options = new ScanOptions();
|
||||
options.setDesiredBarcodeFormats(ScanOptions.QR_CODE);
|
||||
options.setPrompt("Scan server configuration QR code");
|
||||
options.setBeepEnabled(true);
|
||||
options.setBarcodeImageEnabled(false);
|
||||
options.setOrientationLocked(true);
|
||||
return options;
|
||||
}
|
||||
|
||||
/**
|
||||
* Process scan result
|
||||
*
|
||||
* @param contents The scanned QR code contents
|
||||
* @param callback The callback to handle the result
|
||||
*/
|
||||
public static void processScanResult(String contents, QrScanCallback callback) {
|
||||
if (contents == null) {
|
||||
callback.onQrScanCancelled();
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
QrServerConfig config = QrServerConfig.fromQrData(contents);
|
||||
callback.onQrScanned(config);
|
||||
} catch (QrServerConfig.QrConfigException e) {
|
||||
callback.onQrScanError(e.getMessage());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,28 +1,38 @@
|
||||
package com.mattintech.simplelogupload.view;
|
||||
|
||||
import android.content.Intent;
|
||||
import android.content.pm.PackageManager;
|
||||
import android.os.Bundle;
|
||||
import android.text.TextUtils;
|
||||
import android.util.Log;
|
||||
import android.widget.Button;
|
||||
import android.widget.Toast;
|
||||
|
||||
import androidx.activity.result.ActivityResultLauncher;
|
||||
import androidx.appcompat.app.AppCompatActivity;
|
||||
|
||||
import com.google.android.material.textfield.TextInputEditText;
|
||||
import com.journeyapps.barcodescanner.ScanContract;
|
||||
import com.journeyapps.barcodescanner.ScanOptions;
|
||||
import com.journeyapps.barcodescanner.ScanIntentResult;
|
||||
import com.mattintech.simplelogupload.R;
|
||||
import com.mattintech.simplelogupload.constant.AppConstants;
|
||||
import com.mattintech.simplelogupload.model.QrServerConfig;
|
||||
import com.mattintech.simplelogupload.util.PermissionUtil;
|
||||
import com.mattintech.simplelogupload.util.PreferencesUtil;
|
||||
import com.mattintech.simplelogupload.util.QrScannerUtil;
|
||||
|
||||
/**
|
||||
* Activity for server setup
|
||||
*/
|
||||
public class ServerSetupActivity extends AppCompatActivity {
|
||||
private static final String TAG = AppConstants.APP_TAG + "ServerSetupActivity";
|
||||
|
||||
|
||||
private TextInputEditText serverAddressInput;
|
||||
private TextInputEditText serverPortInput;
|
||||
private TextInputEditText clientKeyInput;
|
||||
private ActivityResultLauncher<ScanOptions> qrScannerLauncher;
|
||||
private Button scanQrButton;
|
||||
|
||||
@Override
|
||||
protected void onCreate(Bundle savedInstanceState) {
|
||||
@@ -34,7 +44,17 @@ public class ServerSetupActivity extends AppCompatActivity {
|
||||
serverPortInput = findViewById(R.id.serverPortInput);
|
||||
clientKeyInput = findViewById(R.id.clientKeyInput);
|
||||
Button finishSetupButton = findViewById(R.id.finishSetupButton);
|
||||
|
||||
scanQrButton = findViewById(R.id.scanQrButton);
|
||||
|
||||
// Initialize QR scanner launcher
|
||||
qrScannerLauncher = registerForActivityResult(
|
||||
new ScanContract(),
|
||||
result -> handleQrScanResult(result)
|
||||
);
|
||||
|
||||
// Set up QR scan button click listener
|
||||
scanQrButton.setOnClickListener(v -> initiateQrScan());
|
||||
|
||||
// Set up action bar
|
||||
if (getSupportActionBar() != null) {
|
||||
getSupportActionBar().setTitle("Server Setup");
|
||||
@@ -103,7 +123,81 @@ public class ServerSetupActivity extends AppCompatActivity {
|
||||
Toast.makeText(this, "Error saving settings: " + e.getMessage(), Toast.LENGTH_LONG).show();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Initiate QR code scan
|
||||
*/
|
||||
private void initiateQrScan() {
|
||||
// Check camera permission
|
||||
if (!PermissionUtil.hasCameraPermission(this)) {
|
||||
PermissionUtil.requestCameraPermission(this);
|
||||
return;
|
||||
}
|
||||
|
||||
// Launch scanner
|
||||
ScanOptions options = QrScannerUtil.createScanOptions();
|
||||
qrScannerLauncher.launch(options);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle QR scan result
|
||||
*
|
||||
* @param result The scan result
|
||||
*/
|
||||
private void handleQrScanResult(ScanIntentResult result) {
|
||||
QrScannerUtil.processScanResult(
|
||||
result.getContents(),
|
||||
new QrScannerUtil.QrScanCallback() {
|
||||
@Override
|
||||
public void onQrScanned(QrServerConfig config) {
|
||||
// Auto-fill the input fields
|
||||
serverAddressInput.setText(config.getServer());
|
||||
serverPortInput.setText(String.valueOf(config.getPort()));
|
||||
clientKeyInput.setText(config.getKey());
|
||||
|
||||
Toast.makeText(ServerSetupActivity.this,
|
||||
R.string.qr_scan_success,
|
||||
Toast.LENGTH_SHORT).show();
|
||||
|
||||
Log.d(TAG, "QR code scanned successfully");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onQrScanCancelled() {
|
||||
Toast.makeText(ServerSetupActivity.this,
|
||||
R.string.qr_scan_cancelled,
|
||||
Toast.LENGTH_SHORT).show();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onQrScanError(String errorMessage) {
|
||||
Toast.makeText(ServerSetupActivity.this,
|
||||
getString(R.string.qr_scan_error, errorMessage),
|
||||
Toast.LENGTH_LONG).show();
|
||||
Log.e(TAG, "QR scan error: " + errorMessage);
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle permission request results
|
||||
*/
|
||||
@Override
|
||||
public void onRequestPermissionsResult(int requestCode, String[] permissions, int[] grantResults) {
|
||||
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
|
||||
|
||||
if (requestCode == PermissionUtil.REQUEST_CAMERA_PERMISSION) {
|
||||
if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
|
||||
// Permission granted, initiate scan
|
||||
initiateQrScan();
|
||||
} else {
|
||||
// Permission denied
|
||||
Toast.makeText(this, R.string.camera_permission_denied, Toast.LENGTH_LONG).show();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean onSupportNavigateUp() {
|
||||
onBackPressed();
|
||||
|
||||
@@ -1,28 +1,38 @@
|
||||
package com.mattintech.simplelogupload.view;
|
||||
|
||||
import android.content.pm.PackageManager;
|
||||
import android.os.Bundle;
|
||||
import android.text.TextUtils;
|
||||
import android.util.Log;
|
||||
import android.widget.Button;
|
||||
import android.widget.Toast;
|
||||
|
||||
import androidx.activity.result.ActivityResultLauncher;
|
||||
import androidx.appcompat.app.AppCompatActivity;
|
||||
|
||||
import com.google.android.material.textfield.TextInputEditText;
|
||||
import com.journeyapps.barcodescanner.ScanContract;
|
||||
import com.journeyapps.barcodescanner.ScanOptions;
|
||||
import com.journeyapps.barcodescanner.ScanIntentResult;
|
||||
import com.mattintech.simplelogupload.R;
|
||||
import com.mattintech.simplelogupload.constant.AppConstants;
|
||||
import com.mattintech.simplelogupload.model.QrServerConfig;
|
||||
import com.mattintech.simplelogupload.util.PermissionUtil;
|
||||
import com.mattintech.simplelogupload.util.PreferencesUtil;
|
||||
import com.mattintech.simplelogupload.util.QrScannerUtil;
|
||||
|
||||
/**
|
||||
* Activity for application settings
|
||||
*/
|
||||
public class SettingsActivity extends AppCompatActivity {
|
||||
private static final String TAG = AppConstants.APP_TAG + "SettingsActivity";
|
||||
|
||||
|
||||
private TextInputEditText serverAddressInput;
|
||||
private TextInputEditText serverPortInput;
|
||||
private TextInputEditText clientKeyInput;
|
||||
|
||||
private ActivityResultLauncher<ScanOptions> qrScannerLauncher;
|
||||
private Button scanQrButton;
|
||||
|
||||
@Override
|
||||
protected void onCreate(Bundle savedInstanceState) {
|
||||
super.onCreate(savedInstanceState);
|
||||
@@ -34,7 +44,17 @@ public class SettingsActivity extends AppCompatActivity {
|
||||
clientKeyInput = findViewById(R.id.clientKeyInput);
|
||||
Button saveButton = findViewById(R.id.saveSettingsButton);
|
||||
Button resetButton = findViewById(R.id.resetSettingsButton);
|
||||
|
||||
scanQrButton = findViewById(R.id.scanQrButton);
|
||||
|
||||
// Initialize QR scanner launcher
|
||||
qrScannerLauncher = registerForActivityResult(
|
||||
new ScanContract(),
|
||||
result -> handleQrScanResult(result)
|
||||
);
|
||||
|
||||
// Set up QR scan button click listener
|
||||
scanQrButton.setOnClickListener(v -> initiateQrScan());
|
||||
|
||||
// Set up action bar
|
||||
if (getSupportActionBar() != null) {
|
||||
getSupportActionBar().setDisplayHomeAsUpEnabled(true);
|
||||
@@ -127,7 +147,81 @@ public class SettingsActivity extends AppCompatActivity {
|
||||
loadSettings();
|
||||
Toast.makeText(this, "Settings reset to defaults", Toast.LENGTH_SHORT).show();
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Initiate QR code scan
|
||||
*/
|
||||
private void initiateQrScan() {
|
||||
// Check camera permission
|
||||
if (!PermissionUtil.hasCameraPermission(this)) {
|
||||
PermissionUtil.requestCameraPermission(this);
|
||||
return;
|
||||
}
|
||||
|
||||
// Launch scanner
|
||||
ScanOptions options = QrScannerUtil.createScanOptions();
|
||||
qrScannerLauncher.launch(options);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle QR scan result
|
||||
*
|
||||
* @param result The scan result
|
||||
*/
|
||||
private void handleQrScanResult(ScanIntentResult result) {
|
||||
QrScannerUtil.processScanResult(
|
||||
result.getContents(),
|
||||
new QrScannerUtil.QrScanCallback() {
|
||||
@Override
|
||||
public void onQrScanned(QrServerConfig config) {
|
||||
// Auto-fill the input fields
|
||||
serverAddressInput.setText(config.getServer());
|
||||
serverPortInput.setText(String.valueOf(config.getPort()));
|
||||
clientKeyInput.setText(config.getKey());
|
||||
|
||||
Toast.makeText(SettingsActivity.this,
|
||||
R.string.qr_scan_success,
|
||||
Toast.LENGTH_SHORT).show();
|
||||
|
||||
Log.d(TAG, "QR code scanned successfully");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onQrScanCancelled() {
|
||||
Toast.makeText(SettingsActivity.this,
|
||||
R.string.qr_scan_cancelled,
|
||||
Toast.LENGTH_SHORT).show();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onQrScanError(String errorMessage) {
|
||||
Toast.makeText(SettingsActivity.this,
|
||||
getString(R.string.qr_scan_error, errorMessage),
|
||||
Toast.LENGTH_LONG).show();
|
||||
Log.e(TAG, "QR scan error: " + errorMessage);
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle permission request results
|
||||
*/
|
||||
@Override
|
||||
public void onRequestPermissionsResult(int requestCode, String[] permissions, int[] grantResults) {
|
||||
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
|
||||
|
||||
if (requestCode == PermissionUtil.REQUEST_CAMERA_PERMISSION) {
|
||||
if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
|
||||
// Permission granted, initiate scan
|
||||
initiateQrScan();
|
||||
} else {
|
||||
// Permission denied
|
||||
Toast.makeText(this, R.string.camera_permission_denied, Toast.LENGTH_LONG).show();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean onSupportNavigateUp() {
|
||||
onBackPressed();
|
||||
|
||||
39
app/src/main/res/drawable/ic_qr_scan.xml
Normal file
39
app/src/main/res/drawable/ic_qr_scan.xml
Normal file
@@ -0,0 +1,39 @@
|
||||
<vector xmlns:android="http://schemas.android.com/apk/res/android"
|
||||
android:width="24dp"
|
||||
android:height="24dp"
|
||||
android:viewportWidth="24"
|
||||
android:viewportHeight="24">
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M3,11h8V3H3V11zM5,5h4v4H5V5z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M3,21h8v-8H3V21zM5,15h4v4H5V15z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M13,3v8h8V3H13zM19,9h-4V5h4V9z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M19,19h2v2h-2z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M13,13h2v2h-2z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M15,15h2v2h-2z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M13,17h2v2h-2z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M15,19h2v2h-2z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M17,17h2v2h-2z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M17,13h2v2h-2z"/>
|
||||
<path
|
||||
android:fillColor="@android:color/white"
|
||||
android:pathData="M19,15h2v2h-2z"/>
|
||||
</vector>
|
||||
@@ -88,6 +88,27 @@
|
||||
|
||||
</com.google.android.material.textfield.TextInputLayout>
|
||||
|
||||
<!-- QR Code Scan Button -->
|
||||
<Button
|
||||
android:id="@+id/scanQrButton"
|
||||
style="?attr/materialButtonOutlinedStyle"
|
||||
android:layout_width="match_parent"
|
||||
android:layout_height="wrap_content"
|
||||
android:layout_marginTop="24dp"
|
||||
android:text="@string/scan_qr_code_button"
|
||||
android:drawableLeft="@drawable/ic_qr_scan"
|
||||
android:drawablePadding="8dp"
|
||||
android:paddingVertical="12dp" />
|
||||
|
||||
<TextView
|
||||
android:layout_width="match_parent"
|
||||
android:layout_height="wrap_content"
|
||||
android:layout_marginTop="8dp"
|
||||
android:text="@string/scan_qr_code_hint"
|
||||
android:textAlignment="center"
|
||||
android:textSize="12sp"
|
||||
android:textStyle="italic" />
|
||||
|
||||
<TextView
|
||||
android:id="@+id/testConnectionInfo"
|
||||
android:layout_width="match_parent"
|
||||
|
||||
@@ -67,6 +67,18 @@
|
||||
android:inputType="textPassword" />
|
||||
</com.google.android.material.textfield.TextInputLayout>
|
||||
|
||||
<!-- QR Code Scan Button -->
|
||||
<Button
|
||||
android:id="@+id/scanQrButton"
|
||||
style="?attr/materialButtonOutlinedStyle"
|
||||
android:layout_width="match_parent"
|
||||
android:layout_height="wrap_content"
|
||||
android:layout_marginTop="16dp"
|
||||
android:layout_marginBottom="8dp"
|
||||
android:text="@string/scan_qr_code_button"
|
||||
android:drawableLeft="@drawable/ic_qr_scan"
|
||||
android:drawablePadding="8dp" />
|
||||
|
||||
<Button
|
||||
android:id="@+id/saveSettingsButton"
|
||||
android:layout_width="match_parent"
|
||||
|
||||
@@ -67,4 +67,13 @@
|
||||
<string name="expiry_time_note">Maximum 24 hours (1 day)</string>
|
||||
<string name="cancel_button">Cancel</string>
|
||||
<string name="upload_button">Upload</string>
|
||||
|
||||
<!-- QR Code Scanning -->
|
||||
<string name="scan_qr_code_button">Scan QR Code</string>
|
||||
<string name="scan_qr_code_hint">Scan a QR code from your server admin panel to auto-fill settings</string>
|
||||
<string name="qr_scan_success">Configuration loaded from QR code</string>
|
||||
<string name="qr_scan_cancelled">QR code scan cancelled</string>
|
||||
<string name="qr_scan_error">QR code error: %1$s</string>
|
||||
<string name="camera_permission_required">Camera permission is required to scan QR codes</string>
|
||||
<string name="camera_permission_denied">Camera permission denied. Please enable it in Settings to scan QR codes.</string>
|
||||
</resources>
|
||||
|
||||
141
docs/CHUNKED_UPLOAD.md
Normal file
141
docs/CHUNKED_UPLOAD.md
Normal file
@@ -0,0 +1,141 @@
|
||||
# Chunked Upload Implementation
|
||||
|
||||
This document describes the chunked upload implementation added to the SimpleLogUpload Android application to solve issues with large file uploads on mobile networks.
|
||||
|
||||
## Overview
|
||||
|
||||
The chunked upload system splits large files into smaller pieces (chunks), uploads them individually, and then reassembles them on the server. This approach provides several benefits:
|
||||
|
||||
1. Better reliability on unstable networks (especially 5G/mobile)
|
||||
2. Ability to resume interrupted uploads
|
||||
3. More accurate progress reporting
|
||||
4. Adaptability to different network conditions
|
||||
|
||||
## Components
|
||||
|
||||
### 1. ChunkedUploadManager
|
||||
|
||||
The core component that handles:
|
||||
- Splitting files into chunks
|
||||
- Managing parallel uploads
|
||||
- Tracking upload progress
|
||||
- Handling network changes
|
||||
- Implementing retry logic
|
||||
- Verifying chunk status with server
|
||||
- Automatically retrying missing chunks
|
||||
|
||||
```java
|
||||
ChunkedUploadManager uploadManager = new ChunkedUploadManager(
|
||||
context, file, maxDownloads, expiryHours);
|
||||
```
|
||||
|
||||
### 2. UploadStrategySelector
|
||||
|
||||
Determines whether to use chunked uploads based on:
|
||||
- File size
|
||||
- Network type (WiFi vs. Mobile)
|
||||
- Configurable thresholds
|
||||
|
||||
```java
|
||||
boolean useChunked = UploadStrategySelector.shouldUseChunkedUpload(context, file);
|
||||
```
|
||||
|
||||
### 3. Modified UploadController
|
||||
|
||||
The existing UploadController now:
|
||||
- Selects between standard and chunked uploads
|
||||
- Handles both upload methods with a unified API
|
||||
- Provides progress updates
|
||||
|
||||
## API Requirements
|
||||
|
||||
The server implements four endpoints:
|
||||
|
||||
1. `/upload/start` - Initializes a chunked upload session
|
||||
- Accepts: filename, filesize, chunk_size, total_chunks, max_downloads, expiry_hours
|
||||
- Returns: upload_id
|
||||
|
||||
2. `/upload/chunk` - Handles individual chunk uploads
|
||||
- Accepts: upload_id, chunk_index, total_chunks, chunk (binary)
|
||||
- Returns: success/failure
|
||||
|
||||
3. `/upload/verify` - Verifies which chunks have been received
|
||||
- Accepts: upload_id
|
||||
- Returns: received_chunks, missing_chunks, total_chunks
|
||||
|
||||
4. `/upload/complete` - Finalizes an upload by reassembling chunks
|
||||
- Accepts: upload_id, filename, total_chunks
|
||||
- Returns: upload code (same as standard upload)
|
||||
|
||||
## Configuration
|
||||
|
||||
The chunked upload system is configured with the following parameters:
|
||||
|
||||
- WiFi chunk size: 10MB
|
||||
- Mobile chunk size: 2MB
|
||||
- WiFi threshold: 50MB (files larger than this use chunked upload)
|
||||
- Mobile threshold: 10MB
|
||||
- WiFi parallel uploads: 3
|
||||
- Mobile parallel uploads: 1
|
||||
|
||||
These values can be adjusted in the `ChunkedUploadManager` and `UploadStrategySelector` classes.
|
||||
|
||||
## Enhanced Reliability Features
|
||||
|
||||
### Chunk Verification
|
||||
|
||||
The system implements robust verification to ensure all chunks are properly uploaded:
|
||||
|
||||
1. Client-side tracking of all uploaded chunks by index
|
||||
2. Server endpoint to verify which chunks have been received
|
||||
3. Automatic reupload of any missing chunks
|
||||
4. Verification that files actually exist on disk before finalizing
|
||||
|
||||
### Network Handling
|
||||
|
||||
The implementation includes robust network handling:
|
||||
|
||||
- Detects network type (WiFi/Mobile/5G/LTE)
|
||||
- Adjusts chunk size and parallel uploads accordingly
|
||||
- Monitors network state changes
|
||||
- Pauses uploads on network loss
|
||||
- Resumes uploads when network is restored
|
||||
- Adapts to network type changes during upload
|
||||
|
||||
### Progressive Retry
|
||||
|
||||
Implements intelligent retry logic:
|
||||
|
||||
- Specific retry of only failed chunks
|
||||
- Server-client synchronization of chunk status
|
||||
- Automatic recovery from partial upload failures
|
||||
- Verification before finalization to catch missing chunks
|
||||
|
||||
## Testing
|
||||
|
||||
To test this implementation:
|
||||
|
||||
1. Ensure the server has the required endpoints implemented
|
||||
2. Test with large files (100MB+) on both WiFi and mobile networks
|
||||
3. Test with network transitions (WiFi -> Mobile)
|
||||
4. Test interrupting uploads and resuming them
|
||||
5. Compare performance with the previous implementation
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- Check logs for entries from `ChunkedUploadManager` and `UploadController`
|
||||
- Look for network-related logs to identify connection issues
|
||||
- Check server logs for chunk verification and missing chunk information
|
||||
- Verify server-side chunk handling is working correctly
|
||||
|
||||
## Implementation Status
|
||||
|
||||
- ✅ Basic chunked upload system
|
||||
- ✅ Network type detection and adaptivity
|
||||
- ✅ Chunk verification and automatic retry
|
||||
- ✅ Resume capability for interrupted uploads
|
||||
- ✅ Parallel chunk uploading
|
||||
- ✅ Network state monitoring
|
||||
- ✅ Progressive backoff and retry strategy
|
||||
- ✅ Enhanced progress tracking
|
||||
|
||||
316
docs/LARGE_UPLOAD.md
Normal file
316
docs/LARGE_UPLOAD.md
Normal file
@@ -0,0 +1,316 @@
|
||||
# Large File Upload Analysis and Improvements
|
||||
|
||||
## Current Implementation Analysis
|
||||
|
||||
### Client Side (Android)
|
||||
|
||||
The SimpleLogUpload Android application is designed to upload large log files (up to 220MB) to a simple file upload server. The current implementation uses:
|
||||
|
||||
- OkHttp3 for HTTP requests
|
||||
- A single monolithic file upload in one HTTP request
|
||||
- Fixed network timeouts:
|
||||
- Connection timeout: 60 seconds
|
||||
- Write timeout: 180 seconds (3 minutes)
|
||||
- Read timeout: 60 seconds
|
||||
- Basic retry on connection failure
|
||||
|
||||
Behavior by network type:
|
||||
- **WiFi**: Uploads work successfully
|
||||
- **5G/Mobile**: Socket timeouts occur during upload (`java.net.SocketException: Socket closed`)
|
||||
|
||||
### Server Side (Python/Flask)
|
||||
|
||||
The server is a simple Flask application with:
|
||||
- Standard file upload endpoint (`/upload`)
|
||||
- No explicit chunked upload support
|
||||
- Uses Gunicorn as WSGI server without specific timeout configurations
|
||||
- No specific optimizations for large files
|
||||
|
||||
## Problem Root Causes
|
||||
|
||||
1. **Mobile Network Limitations**:
|
||||
- Mobile carriers may have connection restrictions for large uploads
|
||||
- 5G networks may have more aggressive connection management
|
||||
- Network fluctuations are more common in mobile networks
|
||||
|
||||
2. **Timeout Configuration**:
|
||||
- Current timeout settings may be insufficient for large files over mobile networks
|
||||
- Server and client timeouts aren't aligned
|
||||
|
||||
3. **Single Upload Approach**:
|
||||
- The entire 220MB file is uploaded in a single HTTP request
|
||||
- No resume capability if connection drops
|
||||
- No progress tracking or partial file recovery
|
||||
|
||||
4. **Missing Network Adaptability**:
|
||||
- No differentiation between WiFi and mobile network settings
|
||||
- No adaptive timeout or retry mechanisms
|
||||
- No connection monitoring during upload
|
||||
|
||||
## Recommended Solutions
|
||||
|
||||
### 1. Implement Chunked Uploading
|
||||
|
||||
Develop a proper chunked upload system with these features:
|
||||
|
||||
```java
|
||||
// Example chunked upload strategy
|
||||
public class ChunkedUploadManager {
|
||||
// Adjust these based on testing
|
||||
private static final int DEFAULT_CHUNK_SIZE = 5 * 1024 * 1024; // 5MB chunks
|
||||
private static final int MAX_RETRY_ATTEMPTS = 3;
|
||||
private static final int RETRY_DELAY_MS = 1000;
|
||||
|
||||
// File metadata
|
||||
private File file;
|
||||
private int chunkSize;
|
||||
private int totalChunks;
|
||||
private String uploadId;
|
||||
|
||||
// Network state
|
||||
private boolean isWifiConnection;
|
||||
private boolean isPaused;
|
||||
|
||||
// Tracking
|
||||
private int currentChunkIndex;
|
||||
private Set<Integer> uploadedChunks = new HashSet<>();
|
||||
|
||||
public ChunkedUploadManager(File file, boolean isWifiConnection) {
|
||||
this.file = file;
|
||||
this.isWifiConnection = isWifiConnection;
|
||||
|
||||
// Adjust chunk size based on network type
|
||||
this.chunkSize = isWifiConnection ? DEFAULT_CHUNK_SIZE : (DEFAULT_CHUNK_SIZE / 2);
|
||||
|
||||
this.totalChunks = (int) Math.ceil((double) file.length() / chunkSize);
|
||||
}
|
||||
|
||||
// Methods for initialization, chunk upload, tracking, etc.
|
||||
}
|
||||
```
|
||||
|
||||
**Server Changes Implemented**:
|
||||
```python
|
||||
# New endpoint for chunk verification
|
||||
@app.route('/upload/verify', methods=['POST'])
|
||||
@require_client_key
|
||||
def verify_chunks():
|
||||
# Get upload ID from request
|
||||
upload_id = request.form.get('upload_id')
|
||||
|
||||
# Get session metadata
|
||||
session = metadata[upload_id]
|
||||
|
||||
# Calculate missing chunks
|
||||
received_chunks = set(session['received_chunks'])
|
||||
all_chunks = set(range(session['total_chunks']))
|
||||
missing_chunks = list(all_chunks - received_chunks)
|
||||
|
||||
# Verify files exist on disk
|
||||
for chunk_index in range(session['total_chunks']):
|
||||
chunk_path = os.path.join(session['chunk_dir'], f'chunk_{chunk_index}')
|
||||
if not os.path.exists(chunk_path) and chunk_index in received_chunks:
|
||||
# Remove from received_chunks if file is missing
|
||||
session['received_chunks'].remove(chunk_index)
|
||||
|
||||
# Return missing chunks information
|
||||
return jsonify({
|
||||
'total_chunks': session['total_chunks'],
|
||||
'received_chunks': len(session['received_chunks']),
|
||||
'missing_chunks': missing_chunks
|
||||
})
|
||||
```
|
||||
|
||||
### 2. Adaptive Network Settings
|
||||
|
||||
Implement network-aware settings that adjust based on connection type:
|
||||
|
||||
```java
|
||||
private void configureNetworkSettings() {
|
||||
ConnectivityManager cm = (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE);
|
||||
NetworkInfo activeNetwork = cm.getActiveNetworkInfo();
|
||||
boolean isWifi = activeNetwork != null && activeNetwork.getType() == ConnectivityManager.TYPE_WIFI;
|
||||
|
||||
// Adjust settings based on network type
|
||||
if (isWifi) {
|
||||
// WiFi settings - can be more aggressive
|
||||
CONNECTION_TIMEOUT = 60; // seconds
|
||||
WRITE_TIMEOUT = 180; // seconds
|
||||
READ_TIMEOUT = 60; // seconds
|
||||
CHUNK_SIZE = 10_485_760; // 10MB
|
||||
} else {
|
||||
// Mobile network settings - more conservative
|
||||
CONNECTION_TIMEOUT = 30; // seconds
|
||||
WRITE_TIMEOUT = 90; // seconds
|
||||
READ_TIMEOUT = 30; // seconds
|
||||
CHUNK_SIZE = 2_097_152; // 2MB
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Network Monitoring and Handling
|
||||
|
||||
Implement network state monitoring to handle:
|
||||
- Network type changes during upload
|
||||
- Connection losses
|
||||
- Bandwidth fluctuations
|
||||
|
||||
```java
|
||||
private void registerNetworkCallback() {
|
||||
ConnectivityManager cm = (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE);
|
||||
NetworkRequest.Builder builder = new NetworkRequest.Builder();
|
||||
|
||||
cm.registerNetworkCallback(builder.build(), new ConnectivityManager.NetworkCallback() {
|
||||
@Override
|
||||
public void onAvailable(Network network) {
|
||||
// Resume upload if paused
|
||||
resumeUploadIfPaused();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onLost(Network network) {
|
||||
// Pause upload
|
||||
pauseUpload();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onCapabilitiesChanged(Network network, NetworkCapabilities capabilities) {
|
||||
// Adjust strategies based on new network capabilities
|
||||
boolean isUnmetered = capabilities.hasCapability(NetworkCapabilities.NET_CAPABILITY_NOT_METERED);
|
||||
adjustUploadParameters(isUnmetered);
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Progressive Backoff and Retry Strategy
|
||||
|
||||
Implement a more sophisticated retry mechanism:
|
||||
|
||||
```java
|
||||
private void uploadChunkWithRetry(int chunkIndex) {
|
||||
int retryCount = 0;
|
||||
int maxRetries = 5;
|
||||
long backoffMs = 1000; // Start with 1 second
|
||||
|
||||
while (retryCount < maxRetries) {
|
||||
try {
|
||||
// Attempt to upload chunk
|
||||
boolean success = uploadChunk(chunkIndex);
|
||||
if (success) {
|
||||
return; // Success, exit retry loop
|
||||
}
|
||||
} catch (IOException e) {
|
||||
Log.e(TAG, "Chunk upload failed: " + e.getMessage());
|
||||
}
|
||||
|
||||
retryCount++;
|
||||
if (retryCount < maxRetries) {
|
||||
// Calculate backoff with exponential increase and jitter
|
||||
long jitter = (long) (Math.random() * 0.3 * backoffMs);
|
||||
long sleepTime = backoffMs + jitter;
|
||||
|
||||
Log.d(TAG, "Retrying chunk " + chunkIndex + " after " + sleepTime + "ms (attempt " + retryCount + ")");
|
||||
|
||||
try {
|
||||
Thread.sleep(sleepTime);
|
||||
} catch (InterruptedException ie) {
|
||||
Thread.currentThread().interrupt();
|
||||
return;
|
||||
}
|
||||
|
||||
// Increase backoff for next iteration
|
||||
backoffMs = Math.min(backoffMs * 2, 30000); // Cap at 30 seconds
|
||||
}
|
||||
}
|
||||
|
||||
// If we get here, all retries failed
|
||||
Log.e(TAG, "All retry attempts failed for chunk " + chunkIndex);
|
||||
reportChunkFailed(chunkIndex);
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Enhanced Progress Tracking and Resumability
|
||||
|
||||
Implement more granular progress tracking for better user experience and resumability:
|
||||
|
||||
```java
|
||||
public class UploadProgress {
|
||||
private long totalBytes;
|
||||
private long uploadedBytes;
|
||||
private int totalChunks;
|
||||
private Set<Integer> completedChunks;
|
||||
private boolean isComplete;
|
||||
private String sessionId;
|
||||
|
||||
// Persist progress to disk for resumability
|
||||
public void saveProgress() {
|
||||
SharedPreferences prefs = context.getSharedPreferences("upload_progress", Context.MODE_PRIVATE);
|
||||
SharedPreferences.Editor editor = prefs.edit();
|
||||
|
||||
// Store progress data
|
||||
editor.putString("session_" + sessionId + "_chunks", TextUtils.join(",", completedChunks));
|
||||
editor.putLong("session_" + sessionId + "_bytes", uploadedBytes);
|
||||
editor.putBoolean("session_" + sessionId + "_complete", isComplete);
|
||||
|
||||
editor.apply();
|
||||
}
|
||||
|
||||
// Load progress for resuming uploads
|
||||
public static UploadProgress loadProgress(Context context, String sessionId) {
|
||||
// Implementation details
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Server-Side Optimizations
|
||||
|
||||
Modify the server to better handle large file uploads:
|
||||
|
||||
1. Implement explicit chunked upload support
|
||||
2. Configure Gunicorn timeouts appropriately:
|
||||
```
|
||||
gunicorn wsgi:app --bind 0.0.0.0:7777 --timeout 300 --workers 4
|
||||
```
|
||||
3. Add support for resumable uploads with progress tracking
|
||||
4. Implement temporary file cleanup for abandoned uploads
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
1. **First Phase**:
|
||||
- Implement basic chunked uploading
|
||||
- Add network type detection and adaptive settings
|
||||
- Configure server for higher timeouts
|
||||
|
||||
2. **Second Phase**:
|
||||
- Add upload resumability
|
||||
- Implement network state monitoring
|
||||
- Enhanced progress tracking
|
||||
- Progressive retry mechanism
|
||||
|
||||
3. **Final Phase**:
|
||||
- Server-side optimizations for chunked uploads
|
||||
- Background upload functionality
|
||||
- Bandwidth control options
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. Test uploading the same 220MB file under various network conditions:
|
||||
- Stable WiFi
|
||||
- Congested WiFi
|
||||
- 5G with strong signal
|
||||
- 5G with weak signal
|
||||
- 4G fallback
|
||||
- Network transitions (WiFi → Mobile)
|
||||
|
||||
2. Measure and compare:
|
||||
- Success rate of uploads
|
||||
- Total upload time
|
||||
- Battery consumption
|
||||
- Data usage efficiency
|
||||
|
||||
## Conclusion
|
||||
|
||||
The socket timeout issues on 5G networks stem from trying to upload large files in a single HTTP request without adapting to the characteristics of mobile networks. By implementing chunked uploads with resume capability, network-aware configurations, and robust error handling, we can create a more reliable upload solution for all network types.
|
||||
|
||||
The most critical issue to address is the lack of chunking, which leaves the application vulnerable to connection interruptions, particularly on mobile networks where connections may be less stable than WiFi.
|
||||
@@ -1,6 +1,11 @@
|
||||
android.suppressUnsupportedCompileSdk=36
|
||||
android.useAndroidX=true
|
||||
|
||||
# Enable Gradle toolchain auto-detection and auto-download
|
||||
org.gradle.java.installations.auto-detect=true
|
||||
org.gradle.java.installations.auto-download=true
|
||||
org.gradle.java.installations.paths=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home
|
||||
|
||||
# JVM arguments for kapt with JDK 9+
|
||||
org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8 \
|
||||
--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
|
||||
|
||||
Reference in New Issue
Block a user