Package org.apache.cassandra.spark.bulkwriter

Class BroadcastableClusterInfoGroup

java.lang.Object

org.apache.cassandra.spark.bulkwriter.BroadcastableClusterInfoGroup

All Implemented Interfaces:

java.io.Serializable, MultiClusterSupport<IBroadcastableClusterInfo>, IBroadcastableClusterInfo

public final class BroadcastableClusterInfoGroup
extends java.lang.Object
implements IBroadcastableClusterInfo, MultiClusterSupport<IBroadcastableClusterInfo>

Broadcastable wrapper for coordinated writes with ZERO transient fields to optimize Spark broadcasting.

This class wraps multiple BroadcastableCluster instances for multi-cluster scenarios. Pre-computed values (partitioner, lowestCassandraVersion) are extracted from CassandraClusterInfoGroup on the driver to avoid duplicating aggregation/validation logic on executors.

Why ZERO transient fields matters:
Spark's SizeEstimator uses reflection to estimate object sizes before broadcasting. Each transient field forces SizeEstimator to inspect the field's type hierarchy, which is expensive. Logger references are particularly costly due to their deep object graphs (appenders, layouts, contexts). By eliminating ALL transient fields and Logger references, we:

• Minimize SizeEstimator reflection overhead during broadcast preparation
• Reduce broadcast variable serialization size
• Avoid accidental serialization of non-serializable objects

See Also:

Serialized Form

Method Summary

Modifier and Type	Method	Description
java.lang.String	clusterId()	ID string that can uniquely identify a cluster.
void	forEach(java.util.function.BiConsumer<java.lang.String,IBroadcastableClusterInfo> action)	Iterate through all values
static BroadcastableClusterInfoGroup	from(CassandraClusterInfoGroup source, BulkSparkConf conf)	Creates a BroadcastableClusterInfoGroup from a source ClusterInfo group.
BulkSparkConf	getConf()
java.lang.String	getLowestCassandraVersion()
org.apache.cassandra.spark.data.partitioner.Partitioner	getPartitioner()
IBroadcastableClusterInfo	getValueOrNull(java.lang.String clusterId)	Look up a value based on clusterId
ClusterInfo	reconstruct()	Reconstructs a full ClusterInfo instance from this broadcastable data on executors.
int	size()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.cassandra.spark.bulkwriter.cloudstorage.coordinated.MultiClusterSupport

getValueOrThrow

Method Detail

from

public static BroadcastableClusterInfoGroup from(@NotNull
                                                 CassandraClusterInfoGroup source,
                                                 @NotNull
                                                 BulkSparkConf conf)

Creates a BroadcastableClusterInfoGroup from a source ClusterInfo group. Extracts pre-computed values (partitioner, lowestCassandraVersion) from the source to avoid duplicating aggregation/validation logic on executors.

Parameters:

source - the source CassandraClusterInfoGroup

conf - the BulkSparkConf needed to connect to Sidecar on executors

getConf

@NotNull
public BulkSparkConf getConf()

Specified by:

getConf in interface IBroadcastableClusterInfo

Returns:

the BulkSparkConf configuration needed to reconstruct ClusterInfo on executors

getLowestCassandraVersion

public java.lang.String getLowestCassandraVersion()

Specified by:

getLowestCassandraVersion in interface IBroadcastableClusterInfo

Returns:

the lowest Cassandra version in the cluster

getPartitioner

public org.apache.cassandra.spark.data.partitioner.Partitioner getPartitioner()

Specified by:

getPartitioner in interface IBroadcastableClusterInfo

Returns:

the partitioner used by the cluster

clusterId

public java.lang.String clusterId()

Description copied from interface: IBroadcastableClusterInfo

ID string that can uniquely identify a cluster. When writing to a single cluster, this may be null. When in coordinated write mode (writing to multiple clusters), this must return a unique string.

Specified by:

clusterId in interface IBroadcastableClusterInfo

Returns:

cluster id string, null if absent

size

public int size()

Specified by:

size in interface MultiClusterSupport<IBroadcastableClusterInfo>

Returns:

the total number of clusters

forEach

public void forEach(java.util.function.BiConsumer<java.lang.String,IBroadcastableClusterInfo> action)

Description copied from interface: MultiClusterSupport

Iterate through all values

Specified by:

forEach in interface MultiClusterSupport<IBroadcastableClusterInfo>

Parameters:

action - function to consume the values

getValueOrNull

@Nullable
public IBroadcastableClusterInfo getValueOrNull(@NotNull
                                                java.lang.String clusterId)

Description copied from interface: MultiClusterSupport

Look up a value based on clusterId

Specified by:

getValueOrNull in interface MultiClusterSupport<IBroadcastableClusterInfo>

Parameters:

clusterId - cluster id

Returns:

the value of type T associated with the clusterId, or null if not found

reconstruct

public ClusterInfo reconstruct()

Description copied from interface: IBroadcastableClusterInfo

Reconstructs a full ClusterInfo instance from this broadcastable data on executors. Each implementation knows how to reconstruct itself into the appropriate ClusterInfo type. This allows adding new broadcastable types without modifying the reconstruction logic in AbstractBulkWriterContext.

Specified by:

reconstruct in interface IBroadcastableClusterInfo

Returns:

reconstructed ClusterInfo (CassandraClusterInfo or CassandraClusterInfoGroup)