Class

com.swoop.spark.records

RecordBuilder

Related Doc: package records

Permalink

abstract class RecordBuilder[Input, Data <: Product, Rec <: Record[Data, _], Ctx <: JobContext] extends RecordBuilderLike[Rec, Ctx] with BuildContext[Ctx]

Base class for record builders of single records.

Input

The source input

Data

The record data

Rec

The record

Ctx

The job context

Self Type
RecordBuilder[Input, Data, Rec, Ctx]
Linear Supertypes
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. RecordBuilder
  2. BuildContext
  3. HasDetails
  4. HasFeatures
  5. RecordBuilderLike
  6. ThrowsErrors
  7. HasJobContext
  8. BuildsRecords
  9. Serializable
  10. Serializable
  11. AnyRef
  12. Any
Implicitly
  1. by any2stringadd
  2. by StringFormat
  3. by Ensuring
  4. by ArrowAssoc
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new RecordBuilder(input: Input, jobContext: Ctx)

    Permalink

    input

    source data

    jobContext

    the job context

Abstract Value Members

  1. abstract def buildData: Option[Data]

    Permalink

    Returns the data for the record or None if no record should be generated for the data.

    Returns the data for the record or None if no record should be generated for the data.

    Attributes
    protected
  2. abstract def dataRecord(data: Data, issues: Seq[Issue]): Rec

    Permalink

    Returns a data record for the data.

    Returns a data record for the data.

    Attributes
    protected
  3. abstract def errorRecord(issues: Seq[Issue]): Rec

    Permalink

    Builds an error record.

    Builds an error record. Should never throw an exception.

    Attributes
    protected
    Definition Classes
    RecordBuilderLike

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. def +(other: String): String

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to any2stringadd[RecordBuilder[Input, Data, Rec, Ctx]] performed by method any2stringadd in scala.Predef.
    Definition Classes
    any2stringadd
  4. def ->[B](y: B): (RecordBuilder[Input, Data, Rec, Ctx], B)

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to ArrowAssoc[RecordBuilder[Input, Data, Rec, Ctx]] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  5. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  6. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  7. def build: TraversableOnce[Rec]

    Permalink

    Returns the record built from the input or an empty traversable if the input should be skipped.

    Returns the record built from the input or an empty traversable if the input should be skipped.

    Definition Classes
    RecordBuilderBuildsRecords
  8. def captureData: Option[Data]

    Permalink

    Returns the result of buildData() after saving it in _maybeData for possible use in errorRecord().

    Returns the result of buildData() after saving it in _maybeData for possible use in errorRecord().

    Attributes
    protected
  9. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  10. def collectAnyRecordStatistics(record: Rec): Unit

    Permalink

    Called for every record that is emitted.

    Called for every record that is emitted.

    record

    The record to be emitted.

    Attributes
    protected
  11. def collectDataRecordStatistics(record: Rec): Unit

    Permalink

    Called when a record with non-empty data that should not be skipped is generated.

    Called when a record with non-empty data that should not be skipped is generated.

    record

    The record with data.

    Attributes
    protected
  12. def collectEmptyRecordStatistics(features: Int): Unit

    Permalink

    Called when record builder produced no data from the record input.

    Called when record builder produced no data from the record input. Note that this is different from the case of data being produced and the builder deciding that the record should be skipped.

    features

    The feature mask accumulated by the builder to this point.

    Attributes
    protected
  13. def collectErrorRecordStatistics(record: Rec): Unit

    Permalink

    Called when an error record was produced.

    Called when an error record was produced. The record will not have data.

    record

    The error record.

    Attributes
    protected
  14. def collectInputStatistics(): Unit

    Permalink

    Called once for each top-level build() call for AnyRecordBuilder.

    Called once for each top-level build() call for AnyRecordBuilder.

    Attributes
    protected
    Definition Classes
    RecordBuilderLike
  15. def collectIssueStats(record: Rec): Unit

    Permalink

    Called every time a record is emitted, independent of whether it has data or not and whether it is an error or not.

    Called every time a record is emitted, independent of whether it has data or not and whether it is an error or not.

    Will not be called on skipped records.

    record

    The record about to be emitted.

    Attributes
    protected
    Definition Classes
    RecordBuilderLike
  16. def collectSkippedRecordStatistics(maybeData: Option[Data]): Unit

    Permalink

    Called when data was produced that should be skipped.

    Called when data was produced that should be skipped.

    maybeData

    The data to be skipped or None if no data was produced.

    Attributes
    protected
  17. def collectStatistics(maybeRecord: Option[Rec]): Unit

    Permalink

    Orchestrates statistics collection.

    Orchestrates statistics collection.

    Attributes
    protected
  18. def collectUnhandledExceptionStatistics(record: Rec, ex: Throwable): Unit

    Permalink

    Called when there is an internal error as part of record building.

    Called when there is an internal error as part of record building.

    An error issue will be created for this exception and collectIssueStats will take it into account. This method is called in order to specifically capture statistics about this internal error.

    record

    The record with the unhandled exception in issues.

    ex

    The unhandled exception.

    Attributes
    protected
    Definition Classes
    RecordBuilderLike
  19. def containsSomeFeatures(mask: Int): Boolean

    Permalink

    Returns true if the feature mask collected during record building contains one of more of the 1 bits in the provided bitmask.

    Returns true if the feature mask collected during record building contains one of more of the 1 bits in the provided bitmask.

    Definition Classes
    RecordBuilderHasFeatures
  20. def debug(msg: String, detail: String, id: Int = Issue.Debug.UNKNOWN): Unit

    Permalink
    Definition Classes
    HasDetails
  21. def debug(msg: String, id: Int): Unit

    Permalink
    Definition Classes
    HasDetails
  22. def debug(msg: String): Unit

    Permalink
    Definition Classes
    HasDetails
  23. def ensuring(cond: (RecordBuilder[Input, Data, Rec, Ctx]) ⇒ Boolean, msg: ⇒ Any): RecordBuilder[Input, Data, Rec, Ctx]

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to Ensuring[RecordBuilder[Input, Data, Rec, Ctx]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  24. def ensuring(cond: (RecordBuilder[Input, Data, Rec, Ctx]) ⇒ Boolean): RecordBuilder[Input, Data, Rec, Ctx]

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to Ensuring[RecordBuilder[Input, Data, Rec, Ctx]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  25. def ensuring(cond: Boolean, msg: ⇒ Any): RecordBuilder[Input, Data, Rec, Ctx]

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to Ensuring[RecordBuilder[Input, Data, Rec, Ctx]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  26. def ensuring(cond: Boolean): RecordBuilder[Input, Data, Rec, Ctx]

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to Ensuring[RecordBuilder[Input, Data, Rec, Ctx]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  27. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  28. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  29. def error(msg: String, detail: String, id: Int = Issue.Error.UNKNOWN): Unit

    Permalink
    Definition Classes
    HasDetails
  30. def error(msg: String, id: Int): Unit

    Permalink
    Definition Classes
    HasDetails
  31. def error(msg: String): Unit

    Permalink
    Definition Classes
    HasDetails
  32. def features: Int

    Permalink

    Returns the feature mask collected during record building.

    Returns the feature mask collected during record building.

    Definition Classes
    RecordBuilderHasFeatures
  33. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  34. def formatted(fmtstr: String): String

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to StringFormat[RecordBuilder[Input, Data, Rec, Ctx]] performed by method StringFormat in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  35. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  36. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  37. def info(msg: String, detail: String, id: Int = Issue.Info.UNKNOWN): Unit

    Permalink
    Definition Classes
    HasDetails
  38. def info(msg: String, id: Int): Unit

    Permalink
    Definition Classes
    HasDetails
  39. def info(msg: String): Unit

    Permalink
    Definition Classes
    HasDetails
  40. val input: Input

    Permalink

    source data

  41. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  42. def jc: Ctx

    Permalink

    Shorthand for jobContext.

    Shorthand for jobContext.

    Definition Classes
    HasJobContext
  43. val jobContext: Ctx

    Permalink

    the job context

    the job context

    Definition Classes
    RecordBuilderHasJobContext
  44. def maybeData: Option[Data]

    Permalink

    Returns _maybeData.

    Returns _maybeData.

    Attributes
    protected
  45. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  46. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  47. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  48. def notifyDetail(ex: Throwable): Unit

    Permalink

    Notifies the builder to add this Throwable to the list from which issues will be generated.

    Notifies the builder to add this Throwable to the list from which issues will be generated.

    If the provided parameter is extends FeatureMask, its features will be added to the set aggregated by the builder.

    Definition Classes
    RecordBuilderHasDetails
  49. def notifyFeatures(mask: Int): Unit

    Permalink

    Notifies the builder to add the following features to the aggregate features.

    Notifies the builder to add the following features to the aggregate features.

    Definition Classes
    RecordBuilderHasFeatures
  50. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  51. def throwError(message: String, messageId: Int, cause: Throwable): Nothing

    Permalink
    Definition Classes
    ThrowsErrors
  52. def throwError(message: String, detail: String, messageId: Int, cause: Throwable): Nothing

    Permalink
    Definition Classes
    ThrowsErrors
  53. def throwError(message: String, messageId: Int): Nothing

    Permalink
    Definition Classes
    ThrowsErrors
  54. def throwError(message: String, detail: String, messageId: Int): Nothing

    Permalink
    Definition Classes
    ThrowsErrors
  55. def throwError(message: String): Nothing

    Permalink
    Definition Classes
    ThrowsErrors
  56. def throwError(message: String, detail: String): Nothing

    Permalink
    Definition Classes
    ThrowsErrors
  57. def toRecord(maybeData: Option[Data]): Option[Rec]

    Permalink

    Returns a record from the data or None, if no record should be emitted.

    Returns a record from the data or None, if no record should be emitted. For a record to be emitted, there must either be data or an error and The Features.SKIP flag must not be set in features.

    Attributes
    protected
  58. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  59. def unhandledException(ex: Throwable): Rec

    Permalink

    Returns an error record for an unhandled exception.

    Returns an error record for an unhandled exception. Collects unhandled error metrics.

    Attributes
    protected
    Definition Classes
    RecordBuilderLike
  60. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  61. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  62. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  63. def warn(msg: String, detail: String, id: Int = Issue.Warning.UNKNOWN): Unit

    Permalink
    Definition Classes
    HasDetails
  64. def warn(msg: String, id: Int): Unit

    Permalink
    Definition Classes
    HasDetails
  65. def warn(msg: String): Unit

    Permalink
    Definition Classes
    HasDetails
  66. def [B](y: B): (RecordBuilder[Input, Data, Rec, Ctx], B)

    Permalink
    Implicit information
    This member is added by an implicit conversion from RecordBuilder[Input, Data, Rec, Ctx] to ArrowAssoc[RecordBuilder[Input, Data, Rec, Ctx]] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Inherited from BuildContext[Ctx]

Inherited from HasDetails

Inherited from HasFeatures

Inherited from RecordBuilderLike[Rec, Ctx]

Inherited from ThrowsErrors

Inherited from HasJobContext[Ctx]

Inherited from BuildsRecords[Rec]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from RecordBuilder[Input, Data, Rec, Ctx] to any2stringadd[RecordBuilder[Input, Data, Rec, Ctx]]

Inherited by implicit conversion StringFormat from RecordBuilder[Input, Data, Rec, Ctx] to StringFormat[RecordBuilder[Input, Data, Rec, Ctx]]

Inherited by implicit conversion Ensuring from RecordBuilder[Input, Data, Rec, Ctx] to Ensuring[RecordBuilder[Input, Data, Rec, Ctx]]

Inherited by implicit conversion ArrowAssoc from RecordBuilder[Input, Data, Rec, Ctx] to ArrowAssoc[RecordBuilder[Input, Data, Rec, Ctx]]

Ungrouped