Abstract: While batch processing of data using LLM has huge advantages, a subset of LLM applications requires synchronous processing. This article provides a solution to synchronously process data at scale using OpenAI API.
Introduction Before we described the solution to process massive amounts of data by using OpenAI Batch API. Batch processing is cheaper and easier to run when you have huge amounts of data. At the same time, if your application cannot wait until the batch process is finished, the only option would be to call OpenAI inference endpoints directly.
Abstract: LLMs usage is expanding into new applications. This article provides a solution to process data at scale using OpenAI Batch API.
Introduction LLM “revolution” continues, which results in LLM to start being used in applications, which were historically performed by humans only. While LLM can be used to successfully solve a large class of tasks, with more coming in the near future, it is crucial to perform foundation work, for example: evaluation of the accuracy of the model outputs, proper data preparation, prompt engineering, model fine-tuning (if needed), etc.
Abstract: With the infrastructure platform growing, the number of secrets in the AWS Secrets Manager increases. This article provides the solution to effectively backing up and restoring them.
Introduction This article continues the work regarding providing the possibility for bulk backing up and restoring SSM parameters and secrets. It has the same motivation and design considerations as in the case of parameters. Here we will mention this information in brief, and in case if you would like to read about it more - you can find more information here.
Abstract: With the growth of the infrastructure platform number of SSM parameters increases. This article provides the solution to effectively backing up and restoring them.
Introduction After the creation of infrastructure in the DR scenario or during regular DR tests, developers spend a sizeable amount of time copy-pasting SSM parameters from existing environments. For example, some applications can have 50 SSM parameters to fill (!). Partial or full automation of this step allows developers to save time and effort, opening the possibility to work on other DR-related activities or to simply finish DR tests earlier.
Abstract: Working with an established infrastructure platform includes performing complex deployments requiring careful planning and execution. Static documents, in this case, are unsuitable and can easily result in production incidents. This challenge can be solved by using a custom interactive runbook solution.
Introduction After the initial creation of the infrastructure in the production environments, all subsequent work is devoted to supporting the platform evolution by performing iterative changes. The requirement to do it with minimum downtime can lead to complex multistep deployment procedures, which require thorough planning to avoid production incidents and unexpected platform downtime.
Abstract: A natural evolution of the growing platform is splitting it into almost isolated pieces of the infrastructure deployed from separate TF repositories. These TF repositories contain a lot of similar code, and having a template makes their creation easy and fast, as well as provides additional benefits.
Introduction Having infrastructure monorepo for the whole platform works well in its early life stage but usually creates challenges during its growth.