Job Summary
We are looking for a Senior OSINT & Web Intelligence Application Developer to build a secure platform supporting authorized cybersecurity fraud investigations. The role focuses on web crawling, public data collection, screenshot capture, metadata preservation, relationship mapping, and secure evidence management. The successful candidate will combine strong software development expertise with a clear understanding of privacy, legal compliance, and ethical use of online information.
Key Responsibilities
- Design, develop, and maintain applications for authorized web data collection and open-source intelligence workflows.
- Build scalable web crawling and data extraction modules for publicly accessible websites, social media pages, forums, marketplaces, and other online sources, subject to applicable laws and client authorization.
- Implement browser automation capabilities for capturing screenshots, page snapshots, timestamps, URLs, and contextual metadata for evidence preservation.
- Develop functionality to map relationships between public posts, user profiles, aliases, domains, URLs, contact information, and other publicly available indicators relevant to fraud investigations.
- Create structured data pipelines to collect, normalize, enrich, index, and store web intelligence data securely.
- Integrate search, filtering, tagging, case management, and reporting capabilities for analysts and investigators.
- Ensure the application maintains full audit trails, chain-of-custody records, access logs, and evidence integrity controls.
- Implement role-based access control, encryption, secure authentication, and other security measures aligned with government-grade requirements.
- Collaborate with cybersecurity analysts, fraud investigators, legal/compliance teams, and government stakeholders to define functional and technical requirements.
- Develop APIs and integrations with third-party intelligence tools, databases, threat intelligence platforms, and internal case management systems.
- Optimize crawling performance while respecting technical, legal, and ethical restrictions, including website terms, rate limits, privacy laws, and authorized scope.
- Monitor and improve application reliability, scalability, and performance.
- Prepare technical documentation, deployment guides, user manuals, and system architecture documentation.
- Support testing, troubleshooting, maintenance, and continuous improvement of the platform.
Qualifications
- Proven experience as a software developer, application developer, backend developer, or full-stack developer.
- Strong experience in web scraping, web crawling, browser automation, or data extraction technologies.
- Hands-on experience with tools and frameworks such as Selenium, Playwright, Puppeteer, Scrapy, Beautiful Soup, Requests, or similar technologies.
- Strong programming skills in one or more languages such as Python, JavaScript, TypeScript, Java, Go, or C#.
- Experience with REST APIs, data processing pipelines, and structured/unstructured data handling.
- Knowledge of databases such as PostgreSQL, MySQL, MongoDB, Elasticsearch, OpenSearch, Neo4j, or similar systems.
- Experience capturing and storing screenshots, HTML snapshots, page metadata, and other digital evidence artifacts.
- Understanding of authentication, authorization, encryption, secure coding, and application security best practices.
- Experience designing scalable and maintainable backend services.
- Strong understanding of data privacy, legal compliance, and ethical use of online information.
- Ability to work with sensitive information in a confidential and professional manner.
- Excellent analytical, problem-solving, and documentation skills.
Preferred Qualifications
- Experience developing OSINT, cyber intelligence, fraud detection, compliance monitoring, or investigation platforms.
- Knowledge of cybersecurity fraud patterns, threat intelligence, digital investigations, or online abuse investigations.
- Experience with entity resolution, link analysis, graph databases, or relationship mapping.
- Experience with machine learning, NLP, or text analytics for classification, entity extraction, sentiment analysis, or anomaly detection.
- Familiarity with digital evidence handling, chain of custody, forensic preservation, and admissibility requirements.
- Experience working with government, law enforcement, public sector, or highly regulated environments.
- Knowledge of cloud platforms such as AWS, Azure, or Google Cloud Platform.
- Experience with containerization and orchestration technologies such as Docker and Kubernetes.
- Experience with CI/CD pipelines, Git, automated testing, and DevSecOps practices.
- Relevant certifications such as CISSP, CEH, OSCP, GIAC, AWS Security, Azure Security, or similar are advantageous.
Technical Skills
- Programming: Python, JavaScript, TypeScript, Java, Go, or C#
- Web Automation: Selenium, Playwright, Puppeteer, Scrapy, Beautiful Soup, Requests
- Databases: PostgreSQL, MongoDB, Elasticsearch, OpenSearch, Neo4j, Redis
- Backend Development: REST APIs, microservices, asynchronous processing, message queues
- Frontend: React, Angular, Vue.js, or similar frameworks would be beneficial
- Security: Secure coding, RBAC, encryption, logging, audit trails, vulnerability management
- Cloud & DevOps: AWS, Azure, GCP, Docker, Kubernetes, CI/CD pipelines
- Data Processing: ETL pipelines, data normalization, indexing, search, metadata extraction
Education
- Bachelor’s degree in Computer Science, Software Engineering, Cybersecurity, Information Systems, or a related field preferred.
- Equivalent professional experience will also be considered.
Experience
- Minimum of [5+] years of professional software development experience.
- Minimum of [3+] years of experience with web scraping, web automation, data extraction, OSINT tooling, or cyber intelligence platforms.
- Experience in cybersecurity, fraud investigation, law enforcement support, or public sector projects is highly desirable.
