Friday, September 11, 2026

 

Mac Cleanup Script: Safely Clean Python and Xcode Storage

If you use a Mac for Python development, iOS development, or both, storage can gradually disappear into caches, build artifacts, and old simulator data.

I created this simple Bash script to check and clean some of the commonly accumulated Python and Xcode development files, while deliberately avoiding deletion of Xcode Archives.

The goal is simple:

Clean up development-related storage without blindly deleting important files.

What This Script Cleans

The script currently checks and cleans the following areas:

Python pip cache

Python packages installed through pip can leave cached files behind.

The script checks:

~/.cache/pip
~/Library/Caches/pip

It uses:

python3 -m pip cache purge

to clear the pip cache when Python 3 is available.

Xcode Derived Data

Xcode generates intermediate build files and other temporary development data under:

~/Library/Developer/Xcode/DerivedData

This directory can become quite large over time.

The script removes the contents of Derived Data:

rm -rf "$DERIVED_DATA"/*

Derived Data is generally rebuildable, so removing it is useful when reclaiming space or troubleshooting build-related issues.

Unavailable iOS Simulators

Xcode can accumulate simulator entries that are no longer available.

The script checks whether simctl is available and then runs:

xcrun simctl delete unavailable

This removes unavailable simulator devices.

What the Script Deliberately Does NOT Delete

One important design choice is that the script does not delete Xcode Archives.

Xcode Archives are stored under:

~/Library/Developer/Xcode/Archives

Archives may contain important application builds, so automatically deleting them could result in losing something that is still needed.

The script therefore only reports their size:

NOTE:
Xcode Archives were NOT deleted.
Archives may contain important app builds.

I prefer this approach over a script that simply searches for large directories and deletes them.

Before-and-After Storage Report

Another useful feature is that the script displays the size of the relevant directories before and after cleanup.

For example:

========== CURRENT STORAGE ==========

Python pip cache:
...
Xcode Derived Data:
...
Xcode Archives:
...
iOS Simulators:
...

After the cleanup, it generates the same report again.

This makes it easy to see whether the cleanup actually recovered meaningful storage.

User Confirmation

The script does not immediately start deleting files.

It asks for confirmation:

read -p "Clean safe caches now? (y/n): " answer

If anything other than y or Y is entered, the script exits:

Cleanup cancelled.

This is a small but important safety mechanism for a script that performs filesystem operations.

Complete Script

Here is the complete version:

#!/bin/bash

# ==========================================
# Mac Python + Xcode Cleanup Script
# ==========================================

echo "=========================================="
echo " MAC CLEANUP - PYTHON & XCODE"
echo "=========================================="
echo ""

# ------------------------------------------
# FUNCTION: Display folder size
# ------------------------------------------
show_size() {
    if [ -d "$1" ]; then
        du -sh "$1" 2>/dev/null
    else
        echo "Not found: $1"
    fi
}

# ==========================================
# BEFORE CLEANUP - STORAGE REPORT
# ==========================================

echo "========== CURRENT STORAGE =========="
echo ""

echo "Python pip cache:"
show_size "$HOME/.cache/pip"
show_size "$HOME/Library/Caches/pip"

echo ""
echo "Xcode Derived Data:"
show_size "$HOME/Library/Developer/Xcode/DerivedData"

echo ""
echo "Xcode Archives:"
show_size "$HOME/Library/Developer/Xcode/Archives"

echo ""
echo "iOS Simulators:"
show_size "$HOME/Library/Developer/CoreSimulator"

echo ""
echo "=========================================="
echo ""

# ==========================================
# ASK FOR CONFIRMATION
# ==========================================

read -p "Clean safe caches now? (y/n): " answer

if [[ "$answer" != "y" && "$answer" != "Y" ]]; then
    echo ""
    echo "Cleanup cancelled."
    exit 0
fi

echo ""
echo "Starting cleanup..."
echo ""

# ==========================================
# PYTHON PIP CACHE
# ==========================================

echo "Cleaning Python pip cache..."

if command -v python3 &> /dev/null; then
    python3 -m pip cache purge
else
    echo "Python3 not found. Skipping pip cache."
fi

echo ""

# ==========================================
# XCODE DERIVED DATA
# ==========================================

echo "Cleaning Xcode Derived Data..."

DERIVED_DATA="$HOME/Library/Developer/Xcode/DerivedData"

if [ -d "$DERIVED_DATA" ]; then
    rm -rf "$DERIVED_DATA"/*
    echo "Xcode Derived Data cleaned."
else
    echo "Derived Data folder not found."
fi

echo ""

# ==========================================
# REMOVE UNAVAILABLE SIMULATORS
# ==========================================

echo "Removing unavailable iOS simulators..."

if xcrun --find simctl >/dev/null 2>&1; then
    xcrun simctl delete unavailable
    echo "Unavailable simulators removed."
else
    echo "simctl is not available. Skipping simulator cleanup."
fi

# ==========================================
# DO NOT DELETE XCODE ARCHIVES
# ==========================================

echo "NOTE:"
echo "Xcode Archives were NOT deleted."
echo "Archives may contain important app builds."

echo ""

# ==========================================
# AFTER CLEANUP REPORT
# ==========================================

echo "========== STORAGE AFTER CLEANUP =========="
echo ""

echo "Python pip cache:"
show_size "$HOME/.cache/pip"
show_size "$HOME/Library/Caches/pip"

echo ""
echo "Xcode Derived Data:"
show_size "$HOME/Library/Developer/Xcode/DerivedData"

echo ""
echo "Xcode Archives:"
show_size "$HOME/Library/Developer/Xcode/Archives"

echo ""
echo "iOS Simulators:"
show_size "$HOME/Library/Developer/CoreSimulator"

echo ""
echo "=========================================="
echo " CLEANUP COMPLETE"
echo "=========================================="


Clean up log before and after

==========================================
 MAC CLEANUP - PYTHON & XCODE
==========================================

========== CURRENT STORAGE ==========

Python pip cache:
Not found: /Users/madhu/.cache/pip
 13M	/Users/madhu/Library/Caches/pip

Xcode Derived Data:
416M	/Users/madhu/Library/Developer/Xcode/DerivedData

Xcode Archives:
Not found: /Users/madhu/Library/Developer/Xcode/Archives

iOS Simulators:
2.5G	/Users/madhu/Library/Developer/CoreSimulator

==========================================


Starting cleanup...

Cleaning Python pip cache...
Files removed: 155 (13.5 MB)
Directories removed: 321

Cleaning Xcode Derived Data...
Xcode Derived Data cleaned.

Removing unavailable iOS simulators...

NOTE:
Xcode Archives were NOT deleted.
Archives may contain important app builds.

========== STORAGE AFTER CLEANUP ==========

Python pip cache:
Not found: /Users/madhu/.cache/pip
4.0K	/Users/madhu/Library/Caches/pip

Xcode Derived Data:
  0B	/Users/madhu/Library/Developer/Xcode/DerivedData

Xcode Archives:
Not found: /Users/madhu/Library/Developer/Xcode/Archives

iOS Simulators:
2.5G	/Users/madhu/Library/Developer/CoreSimulator

==========================================
 CLEANUP COMPLETE
==========================================

How to Use It

Save the script, for example as:

mac-cleanup.sh

Make it executable:

chmod +x mac-cleanup.sh

Then run it:

./mac-cleanup.sh

The script first reports the current storage usage and then asks for confirmation before performing the cleanup.

Why I Kept the Script Conservative

A cleanup script has to balance two things:

  • Recovering storage

  • Avoiding accidental deletion of useful development artifacts

For that reason, this script focuses on areas that are intended to be regenerated or are explicitly identified as unavailable simulator data.

I intentionally left Xcode Archives untouched. They can be much more valuable than the storage they consume.

This makes the script more suitable for regular developer-machine maintenance than an aggressive "delete everything related to Xcode" script.

Final Thoughts

Development environments can quietly consume significant disk space over time.

For a Mac used for both Python and Xcode development, periodically checking caches, Derived Data, and obsolete simulator entries can help keep storage under control.

The important principle is not simply to delete large directories.

Know what you are deleting, understand whether it can be regenerated, and protect anything that may contain valuable application artifacts.





Monday, June 22, 2026

When a Security Recommendation Became an Operational Incident

A database team was implementing recommendations from the latest CIS Benchmark for an Oracle 19c Server.

Among the reviewed items were hidden parameters related to symbolic link handling and directory validation.

The implementation was completed during a scheduled maintenance window.

- Database startup was successful.

- Applications connected successfully.

- All standard validation checks passed.

- The change was signed off as successful.

Monday Morning

The following morning, during routine health checks, an engineer noticed that all scheduled Oracle Data Pump export jobs had failed overnight.

- The database was healthy.

- Applications were working.

- Monitoring showed no major issues.

- Only the export jobs had failed.

Investigation

The first step was to review the Data Pump logs generated by the failed export jobs.

The logs indicated that Data Pump was unable to access the export directory used by the DIRECTORY object.

At this point, the focus shifted from database availability to filesystem access.

The team validated:

  • Oracle DIRECTORY object configuration
  • Directory privileges
  • Windows filesystem permissions
  • Symbolic link configuration
  • Target storage accessibility

The DIRECTORY object was valid.

The symbolic link existed and was accessible from the operating system.

The target storage location was also available.

This created an interesting situation:

Windows could access the path successfully, but Oracle Data Pump could not.

The investigation then turned to the changes implemented during the weekend maintenance window.

The Technical Detail

The Oracle Data Pump DIRECTORY object pointed to a Windows path that contained a symbolic link.

The symbolic link redirected exports to the actual storage location.

This configuration had existed for years and had never caused operational issues.

After reviewing the hardening changes, the team discovered that newly implemented security-related parameters were affecting Oracle's handling of symbolic links.

The result was that Data Pump could no longer use the DIRECTORY object as expected.

The More Interesting Discovery

While reviewing the CIS Benchmark documentation, another observation emerged.

The benchmark did not explicitly state:

"Set this parameter to this value in all environments."

Instead, the recommendation required administrators to evaluate the security implications and determine whether the setting was appropriate for their environment.

In other words, the benchmark required analysis, not blind implementation.

The environment contained a legitimate dependency on symbolic links.

That dependency had not been identified before the change was implemented.

Root Cause

Technically, the export failure was caused by the interaction between Oracle's symbolic link security controls and the existing Data Pump configuration.

From a process perspective, the root cause was different.

The parameter change was implemented without a full assessment of:

  • Existing architecture
  • Filesystem dependencies
  • Oracle Data Pump usage
  • Business impact

The Datapump failure was simply the visible symptom.

Lessons Learned

1. Compliance Recommendations Are Not Always Mandatory Configurations

A benchmark recommendation should trigger analysis.

It should not automatically trigger implementation.

Understanding why a recommendation exists is just as important as understanding how to implement it.

2. Hidden Parameters Deserve Additional Scrutiny

Hidden parameters can influence behavior in ways that are not immediately obvious.

Any change involving hidden parameters should include:

  • Impact assessment
  • Architecture review
  • Functional testing
  • Rollback planning

3. Test Critical Business Functions

Database startup and application connectivity are not enough.

Validation should include:

  • Data Pump exports
  • RMAN backups
  • Batch jobs
  • Replication
  • Filesystem-dependent operations

4. Architecture Assumptions Matter

The symbolic link had existed for years.

Nobody questioned it because it worked.

The change simply exposed a dependency that already existed.

Final Thoughts

One of the easiest mistakes in technology is assuming that every recommendation should be implemented exactly as written.

Benchmarks provide guidance.

Engineering requires judgement.

The most important lesson from this incident was not related to Oracle Data Pump or symbolic links.

It was a reminder that before implementing any security recommendation, we should first ask:

"What problem is this recommendation trying to solve, and what existing assumptions in my environment might it affect?"

Asking that question before implementation could have prevented the incident altogether.

 

Sunday, March 15, 2026

Optimizing Oracle 19c SQL: Key Takeaways

  • Define Clear Tuning Goals: Prioritize what matters – low response time for OLTP vs. high throughput for analytics. Tuning means hitting measurable benchmarks, not guessing.
  • Design for Performance: Build scalability in from day one. Use connection pooling and write queries to “parse once, run many times.” Always use bind variables instead of literals to share cursors and avoid costly hard parses.
  • Know the Optimizer: Oracle’s Cost-Based Optimizer picks plans by estimating cardinality (row counts) and cost (I/O+CPU). Accurate stats (and histograms for skew) are essential. The optimizer’s cost = I/O + CPU + memory, so even small misestimates can flip join types or access paths.
  • Leverage Adaptive 19c Features: Oracle 19c can fix bad plans on-the-fly. For example, it can switch a nested-loop to a hash join mid-execution if row counts are higher than expected. It even re-optimizes SQL automatically if the first execution showed wrong cardinality. In practice this means fewer manual tune cycles – just enable OPTIMIZER_ADAPTIVE_PLANS=TRUE (the default).
  • Keep Statistics Fresh: Let the database do the heavy lifting. Oracle 19c introduces real-time and high-frequency statistics gathering, so the optimizer rarely has stale data. In 19c a light-weight job runs every 15 minutes by default to update stats for tables that changed. Ensure AUTOMATIC_STATISTICS_UPDATE is ON and consider manual FULL gathers when massive data loads occur.
  • Use Automated Advisors: Oracle’s tuning advisors are powerful. The SQL Tuning Advisor can analyze top SQL and even generate SQL Profiles or rewrite recommendations. The SQL Access Advisor will suggest the right indexes, materialized views or partitioning for a workload. Best of all, 19c’s Automatic Indexing can create and test indexes for you, making them visible only if they improve performance.
  • Lock in Good Plans: Enable SQL Plan Baselines (SQL Plan Management) on critical queries to “whitelist” good plans. When a new plan appears, Oracle will only use it if it’s been validated. The new SPM Evolve task in 19c runs frequently to test any new plans behind the scenes and automatically accept the best one. This guards against plan regressions after stats or version changes.
  • Monitor and Diagnose: Keep an eye on queries in real time. Oracle 19c automatically tracks SQL statements that run in parallel or take >5 seconds of CPU/I/O. Use DBMS_SQLTUNE.REPORT_SQL_MONITOR and DBMS_XPLAN to see exactly what’s happening. In 19c, developers can view their own SQL monitor reports without special privileges – a great way to troubleshoot slow queries on the fly.

Practical Tips: Always use bind variables to enable soft parses (reducing library-cache latches). And enable AWR/ADDM (Requires additional license pack) to find top-consuming SQL, then feed those statements to the SQL Tuning Advisor.

Sunday, November 9, 2025

 

Working with Databases Using SQLite in Python

Introduction

Databases are essential for storing, organizing, and retrieving data efficiently. Whether it’s a simple desktop app or a data-driven web application, managing data effectively is key.

If you’re starting your journey with databases or need a lightweight option for your project, SQLite is a great choice. It’s fast, reliable, and comes built into Python; no installation or configuration is required. In this post, we’ll explore how to use SQLite with Python to create, read, update, and delete data with just a few lines of code.


What is SQLite?

SQLite is a self-contained, serverless database engine. Unlike MySQL or PostgreSQL, it doesn’t require a separate server to run your entire database lives in a single .db file.

Key advantages:

  •  Lightweight and fast

  •  No setup required

  •  Ideal for small apps, data analysis, or prototypes

  •  Built directly into Python’s standard library

Step-by-Step Guide: Using SQLite in Python

You can start using it instantly by importing Python’s built-in module:

import sqlite3

1) Create a Connection

To start, connect to a database. If it doesn’t exist, SQLite will create it automatically.

import sqlite3 conn = sqlite3.connect('students.db')

2) Create a Table

Now, let’s create a simple table to store student information.

cursor = conn.cursor() cursor.execute(''' CREATE TABLE IF NOT EXISTS students ( id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, age INTEGER, grade TEXT ) ''') conn.commit()

3) Insert Data

You can easily insert records into the table.

c.execute("insert into customer values ('John','Stathom','C')")


You can easily insert multiple records into the table.

many_students = [
                    ('Mark',16,'A'),
                    ('Steve',13,'D'),
                    ('Mark',15,'B')
                 ]
cursor.execute("INSERT INTO students VALUES (?, ?, ?)",  many_students)

conn.commit()

4) Retrieve Data

Fetching records is simple with SELECT.

cursor.execute("SELECT * FROM students") rows = cursor.fetchall() for row in rows: print(row)

5)Update or Delete Records

Updating or deleting data uses standard SQL commands.

cursor.execute("UPDATE students SET grade = 'A+' WHERE name = 'Alice'") cursor.execute("DELETE FROM students WHERE age < 18") conn.commit()

6) Close the Connection

Always close your connection once you’re done.

conn.close()


Tuesday, June 17, 2025

ScyllaDB vs Cassandra: What Makes ScyllaDB the Better Choice?

For years, Apache Cassandra has been a go-to NoSQL database for applications that demand high availability and horizontal scalability. It’s widely used and trusted by big names in the industry. But as workloads grow and performance becomes critical, many are starting to explore ScyllaDB a newer, high-performance alternative built with modern hardware in mind.

So what makes ScyllaDB stand out from Cassandra? Here's a look at the key differences and why more teams are making the switch.

1. C++ Instead of Java

The most fundamental difference is in the programming language. Cassandra is written in Java, which means it relies on the Java Virtual Machine (JVM) for memory management. This introduces latency issues due to garbage collection and other runtime overhead.

ScyllaDB, on the other hand, is built in C++. It doesn't use a garbage collector, which results in much more consistent performance. That means no random latency spikes or pauses under load, a major win for real-time applications.

2. Asynchronous, Shared-Nothing Design

Cassandra uses a thread-per-request model and shares memory across threads. As clusters grow, this design can become inefficient and harder to manage.

ScyllaDB was built differently. It uses an asynchronous, shared-nothing architecture where each CPU core handles its own data independently. There's no contention between threads, no locks, and no need to wait on other processes. This leads to significantly better performance, especially as the number of cores increases.

3. Thread-Per-Core Efficiency

ScyllaDB automatically distributes data across CPU cores and disks. Each core runs its own thread, dedicated to handling its portion of the data.

This "thread-per-core" model allows ScyllaDB to scale vertically with the number of cores, making full use of modern multi-core processors. It also means fewer context switches and more efficient CPU usage compared to Cassandra.

4. Predictable Latency and Higher Throughput

When benchmarked side by side, ScyllaDB consistently delivers better throughput than Cassandra  often by 3 to 10 times  while maintaining much lower latency.

Where Cassandra might show unpredictable latency, especially under load, ScyllaDB remains stable and responsive. This is a big advantage for applications like recommendation engines, fraud detection, and IoT platforms where every millisecond counts.

5. Easy Migration with CQL Compatibility

ScyllaDB supports the Cassandra Query Language (CQL), which means it’s API-compatible with Cassandra. Most applications that use Cassandra can be pointed to a ScyllaDB cluster with little or no code changes.

There are also tools provided by ScyllaDB, like the Scylla Migrator, that help automate and simplify the migration process.

6. Easier to Manage and Self-Tuning

One of the pain points with Cassandra is the need to tune dozens of JVM settings, compaction strategies, and cache configurations. It often requires deep expertise to operate Cassandra efficiently at scale.

ScyllaDB simplifies all of this. It is largely self-tuning and handles most configuration tasks automatically. This reduces the need for constant tweaking and makes it easier for smaller teams to manage large deployments.

7. Built-In Monitoring and Management Tools

ScyllaDB comes with built-in tools like Scylla Manager for maintenance tasks and a preconfigured monitoring stack based on Prometheus and Grafana. These tools make it easier to keep an eye on cluster health, schedule repairs, and run backups.

Cassandra has monitoring options too, but many require third-party tools or enterprise licensing.

8. Lower Total Cost of Ownership

Because of its efficiency, ScyllaDB typically requires fewer nodes to handle the same workload as Cassandra. That means you can achieve better performance with less infrastructure, which directly translates to cost savings.

In real-world deployments, some companies have reported reducing their cluster size by half or more after switching to ScyllaDB without sacrificing performance.

 

Why Choose ScyllaDB? 

Features                          Apache Cassandra    ScyllaDB                              
Language Java   C++
Garbage Collection Yes (JVM)   No
Architecture Thread-per-request       Shared-nothing, async
CPU Efficiency Moderate   High (thread-per-core)
Latency Variable   Predictable and low
Throughput Good   3x to 10x better
Operational Complexity High   Low
Compatibility CQL   CQL + DynamoDB API
Cost Higher   Lower (fewer nodes needed)

Final Thoughts

Cassandra has served many companies well, and for good reason. But ScyllaDB builds on the same principles and takes them further with better performance, lower latency, easier operations, and a much lower cost of ownership.

If you're already using Cassandra and feeling the limits, or if you're starting a new project that demands speed and scalability, ScyllaDB is worth serious consideration. It’s a modern database for the modern era without the headaches.

 

  Mac Cleanup Script: Safely Clean Python and Xcode Storage If you use a Mac for Python development, iOS development, or both, storage can g...