Freelance, market research, confidential client
Field Operation Platform
- Web
- Android
- Fullstack Engineer
Field agents visit locations for market-research projects, record photo, video and audio evidence, and send it back to the operations team, often from places with a weak signal. Large files go up in 30 MB chunks straight to S3, and an upload that fails partway continues from the next unfinished chunk. Operations staff plan projects, import visit tasks from spreadsheets, assign agents and review every file in a web dashboard.
- Where
- Freelance, for a market-research company that stays unnamed
- Role
- Fullstack Engineer. I built the backend, the Android app and the web dashboard.
- Dates
- Feb 2025 – Aug 2025
- Platforms
- Web (operations dashboard) and Android (field agents)
- Stack
- NestJS 11, TypeScript, MongoDB, Flutter, Nuxt 3, Vue 3, Vite, Vuetify 3, AWS S3, Agenda, Docker
- upload chunks
- 30 MB
- evidence types: photo, video, audio
- 3
- apps built: API, Android, web
- 3
- API modules
- 21
01Context and my role
I built the NestJS API, the Flutter app for Android, and the Nuxt dashboard, plus CI/CD and deployment. I coordinated the project directly with the client. The API started from my own ack-nestjs-boilerplate.
- Feb 2025Main build starts
API, Android app and web dashboard.
- Aug 2025Main build done
02What it does
Offline-first fieldwork
Sign-in needs a connection. After that the visit, the in-app camera (photo and video) and the audio recorder run on the phone with no network. Captures land in a folder per task and wait there. Upload is a later step, when the agent is online.
Presigned uploads to S3
Files up to 30 MB use one presigned URL. Larger files go in 30 MB chunks. The phone puts bytes straight on S3, so the API is not in the data path. Each URL expires after 30 minutes so a leftover link cannot be reused.
Background upload queue
An in-app queue uploads in the background. The agent does not sit on each file. Retry and continue are on the phone. Force-closing the app stops the queue.
Device health check
Before fieldwork the app grades the phone A–D from battery, free disk space and system info, so operations can see whether that device is fit for the project.
Google Drive import
Field work can happen outside the app. Operations start one job that pulls those videos from a Google Drive folder into S3 and attaches them to the visit. Pause, resume and cancel sit on that job.
Operations analytics
A dashboard of project counts by preparation, ongoing and completed, a completion rate, visit and completed totals against the target, and monthly and yearly project timelines.
Admin from start to finish
The Nuxt dashboard runs the whole job: clients and projects, spreadsheet import, assignment, review (approve, reject, comments, map), agent and admin accounts, a storage calculator, and the import queue.
03How it works
As field agent
Capture on the device, upload when connected
- Captured media is saved to a folder per task on the phone, so it survives app restarts and can be uploaded later.
- The app checks connectivity before each API call and tells the agent when the connection is lost.
Chunked, resumable uploads
- Small files in one request: a file up to 30 MB goes up in one request to a presigned S3 URL.
- Large files in 30 MB chunks: the API starts an S3 multipart upload and hands out a presigned URL per part. The phone uploads each chunk straight to S3, keeps the ETag S3 returns, then asks the API to complete the upload and register the file on the task.
- One file at a time: files go through an in-app upload queue with progress, cancel and retry.
- Resume after a failure: if an upload fails after the first chunk, the file is marked paused, and "continue" resumes from the next unfinished chunk.
As operations
API
- 21 modules: 9 project modules (clients, projects, assignment, tasks, task files, comments, field agents, storage usage, dashboard) on 12 shared modules from the boilerplate, including auth, users, roles and policies, API keys, settings, activity log and health.
- A MongoDB replica set through Mongoose, JWT verified against a JWKS key server, CASL permissions, English and Indonesian messages, Swagger docs, spreadsheet import and export, and optional CDN URLs for media.
Mobile app
- Flutter with MVVM: Provider view models, get_it for dependency injection, Dio for HTTP.
- Development, staging and production build variants; Android APKs split per CPU architecture.
Web dashboard
- Nuxt 3 on Vue 3, with TypeScript, Vite and file-based routing on Vue Router (both through Nuxt), Pinia for state, and Vuetify 3 on an admin template.
- CASL permissions in the UI, vue-i18n, ApexCharts and Chart.js, Leaflet maps.
Background import queue
Operations can bring in media that was collected outside the app:
- Bulk import: media from a Google Drive folder is copied into S3 by background jobs on Agenda, a job queue backed by MongoDB.
- Pause, resume and cancel per import: jobs save checkpoints, so a restarted job continues where it stopped.
- Limits and cleanup: logs of failed files, concurrency limits, and a cleanup job for finished imports.
- On the dashboard: a queue page to follow bulk imports, city filters, file renaming and gallery upload for admins.
04Decisions and trade-offs
Upload straight to S3
- Why
- The phone sends media to S3 with presigned URLs, so large videos never pass through the API server.
30 MB chunks
- Why
- A failed chunk costs at most 30 MB of mobile data to send again, not the whole video.
Queue on the phone
- Why
- The agent can leave the upload screen. The queue keeps going until the app is force-closed. On the wire, files still go one at a time so a weak link is not shared across parallel uploads.
Files on disk, not in a database
- Why
- Keeping captures as files per task was enough to survive restarts without adding a local database.
Presigned URLs expire in 30 minutes
- Why
- A URL that sits unused cannot be reused later.
05Testing and delivery
- Quality gates: ESLint, Prettier, commitlint, Husky with lint-staged, cspell and type checks on all three codebases; stylelint on the dashboard. There are no automated tests.
- API CI/CD on GitHub Actions: lint on push, a version-tag release, and a release workflow that builds a Docker image, pushes it to AWS ECR and deploys to AWS EC2 through AWS Systems Manager (SSM), with separate development and production instances.
- Dashboard CI/CD: the same pattern: a multi-stage Docker build, then SSM copies the static build onto the host.
- Mobile: APKs built locally with a script per build variant.
06Stack
- API
Node.js LTS
TypeScript 5.8
NestJS 11
SWC
MongoDB replica set
Mongoose 8
Swagger
- Files
AWS S3
CDN URLs
- Auth and access
JWT
JWKS
CASL
- API keys
- Jobs
Agenda
MongoDB job queue
- Ops
Sentry
Pino
NestJS i18n
- spreadsheet import
- spreadsheet export
- Mobile
Flutter 3.29
Dart 3.7
Provider (MVVM)
get_it
Dio
camera
record
media_kit
geolocator
connectivity_plus
flutter_foreground_task
async_queue
flutter_secure_storage
Sentry
- Web dashboard
Nuxt 3
Vue 3.5
Vite 5
TypeScript 5.7
Vue Router
Pinia
Vuetify 3
CASL
vue-i18n
- ApexCharts
Chart.js
Leaflet
pnpm
- Delivery
Docker
GitHub Actions
AWS ECR
AWS EC2
AWS SSM