Categories
BLOG

dab tree

Dab tree

GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.

GitHub is where the world builds software

Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

Distributed Asynchronous Boosting Tree (DAB Tree)

Team: Qin Lyu, Bingfeng Xia, MingLong Wu, Hengte Lin

The goal of this project is to apply parallel computation techniques taught in Harvard CS205 course to a data science or a computational science problem. In general, by massive parallel computation, an algorithm is expected to accommodate larger datasets or to perform large scale computation efficiently.

Gradient boosting decision tree (GBDT) is a machine learning technique that can be used for regression and classification problem [1]. GBDT was proposed by Jerome H. Friedman [2,3] that constructs additive model using decision tree as a weak learner. Advantages of GBDT include its capability to model feature interaction and to perform inherent feature selection.

GBDT has been applied to wide varities of applications including physics [4], computer vision [5], and web-search ranking [6, 7]. To cope with increased scale of datasets, a distributed and parallel version of GBDT will be implemented in this project. The scaling property of the implemented system will be tested based on a real world application.

Tree the a kind of prediction model based on thresholds. According to WikiPedia, “A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.”

Here is an example:

What is a Gradient Boosting Tree

First One need to know what is Ensamble. Ensamble is a technology to group some models and make them vote for the final result. There are two main kinds of ensamble, bagging and boosting. In bagging, models are indipendent of each other. In Gradient Boosting, models interatively learn what’s left from the previous model.

A distributed asynchronous stochastic gradient boosting system will be implemented and be tested on a HPC cluster. In addition, parallel computing will be used to accelerate training models on each node.

Data can be first divided into packs and saved as google protobuf files. Then protobuf files are read by trainers in batches. Trainers will iteratively grow nodes on a tree until reach limit. Then trainer will read next data batch and train the next tree.

Pipeline of training one tree

Local Training Process

Both data preparation and tree node growing are parallelized using c++ threading.

The algorithm can operate on an OpenMPI cluster. The cluster was designed to have one Master and several Workers. Workers are the nodes that actually read data and create trees, while Master act as a trading center to asynchronously exchange trees among Workers.

Batch Delivery Mode

To reduce the communication overhead while trading trees with Master, DABTREE supports the option to deliever trees in batches. Delivering trees in batches will significantly reduce the cost of communication, with the cost of reducing accuracy.

A Gradient boosting decision tree algorithm is implemented with C++ using serial computation approach. A parallel algorithm using MPI and c++ threading is also implemented. Initial tests are performed by comparing training time to existing software packages including GBoost in sklearn and XGBoost. Computation time is listed below. All algorithms achieve similar accuracies.

Pull the repo in master branch.

cd into “/build” folder.

cd back to root folder.

type “mpirun -n #numnodes ./cluster –round=#numTreePerNode –threads=#threadPerWorker –batch=#samplingSize –bundle=#(true/false)batchFunctionality “

PS: If you used batch delivery functionality, numTreePerNode must be multiple of 10.

Scalability and efficiency of algorithm will be tested. Training speed and prediction accuracy are comparable with an existing Gradient Boosting Tree library.

Training size (88896, 17)

The algorithm is faster than sklearn even using single thread, and achieves multiple times of speedup with multi-threading. We plan to continue developing the repo and make it an open source project. The target is to achieve better speed and accuracy than all existing packages.

Distributed Asynchronous Boosting Tree. Contribute to CrimsonInn/dab-tree development by creating an account on GitHub.

DAB/DAB+ Advanced Monitoring Probe
EdgeProbe Advanced DAB/DAB+

The ideal tool for accurate & cost-effective monitoring of the quality actually delivered to all points of DAB/DAB+ networks.

Applications

  • 24/7 Monitoring and Maintenance of the SFN networks: TX transmission and SFN Overlapping Reception Areas
  • Generation of Service Availability reports for Service Level Agreements
  • Plan and improve the network configuration by identifying global trends

Benefits

  • Standalone, easy to use and configure, fast deployment, SNMP compatible
  • Reduce TX sites maintenance cost by anticipating and identifying issues
  • Increase customer satisfaction by detecting & preventing network degradations before your customers do
  • Remotely accessible, compatible with low bandwidth control networks (GPRS/3G/4G)
  • Low power consumption 25W
  • Key features
  • Technical specifications
  • Options & Warranties

Combined with a Network Monitoring System or not, the EdgeProbe Advanced provides a powerful broadcast network alert & diagnosis tool allowing Digital Radio network operators to monitor global trends and anticipate potential failures.
EdgeProbe Advanced is able to monitor DAB/DAB+ signals at transmitter outputs as well as in the reception areas, through its RF inputs (up to 4 in 1RU).

DAB/DAB+ monitoring through the RF inputs (up to 4 in 1RU)

Signal Level, MER, SNR, FIC BER, MSC BER (Pre/Post, per sub-channel)

Compatible Band III VHF (168 to 240 MHz)

Channel RF Spectrum and Constellation display

RF Shoulder Upper/Lower measurement

Mode I, II support; automatic detection

Local analog audio output: via front panel controls

SFN monitoring at TX or Reception area

  • RF signal time synchronization; detects up to +/- 1.2 seconds time drift

In Reception area (SFN overlapping area):

  • Channel Impulse Response (Echoes), with advanced Echo Pattern mode: better echo in error identification even if the main (stongest) echo suffers changes; no time shift if the main echo disappears
  • TII detection (Main/Sub ID)

DAB Transport monitoring

Ensemble Service Plan: check ensemble structure

Service information (SI)

32 GB of internal storage (up to 4 in 1RU)

Alarm logs & RF parameter trends up to 4 months

CSV format files. Available for download via web GUI or FTP connection (automation scripts)

Internal GNSS receiver (HW option)

Generates an internal 1PPS reference signal for SFN synchronization measurements – which is independent from the modulator’s reference signal

GPS & GLONASS support

Dual Power Supply (HW option)

One additional Power Supply can be installed on the equipment in order to ensure the power redundancy

INTERFACES

    RF Connector In

Up to 4x RF inputs (N-type female 50 Ω)

-80 to -5 dBm; RF lock down to -80dBm

1x GNSS antenna input (SMA-type 50 Ω) (GPS/GLONASS), 3.3V antenna power up

1x 1PPS input (BNC-type female 50 Ω)

1x 10MHz input (BNC-type female 50 Ω)

Up to 4x analogue audio outputs (TRS 3.5mm) – front panel

MONITORING FEATURES

    RF Monitor

Demodulation status: Lock / Unlock

Signal level: -100 to -5 dBm measure range, ± 1 dBm

MER: 0 to 40 dB, ± 1 dB, 0.1 dB resolution

SNR: 0 to 50 dB, ± 1 dB, 0.1 dB resolution

FIC Pre-Viterbi BER

MSC Pre/Post-Biterbi BER (per sub-channel)

Channel RF Spectrum and Constellation display

RF Shoulder Upper/Lower attenuation

CIR – Echoes: TII extraction (Main ID, Sub ID). Validation of the field reception quality.

With TestTree’s unique Echo Pattern mode: better echo in error identification when the main (strongest) echo suffers changes; prevent time shift of all echoes when main echo disappers.

SFN Drift measured at RF level. Allows rapid identification of which TX site is causing SFN issues

Check ensemble structure: service & component information

Service information (SI)

Monitor sequentially (round-robin) multiple frequencies over 1 RF input. Monitoring status & context is kept between two sucessive monitoring rounds

Up to 4x 32 GB of internal storage (per monitoring unit): alarm logs & RF trends up to 4 months. CSV format files. Available for download via web GUI or FTP connection

PHYSICAL

  • Height: 45 mm / 1.7 in, Width: 440 mm / 17.3 in, Depth: 300 mm / 11.8 in
  • Format: 1 RU, width 19”, Power supply: 100-240 VAC +/-10%
  • Power consumption: 25W, Redundant Power Supply (HW option)

ENVIRONMENT

    Operating temp

-20 to 55°C / -4 to 131 °F

-20 to 70°C / -4 to 158°F

0 to 95%, non condensing

EdgeProbe Advanced DAB/DAB+ includes by default:

  • Hardware:
    • 2 monitoring Units 2x (RF N-type 50 Ω in) & 1x IP Control in 1 RU
    • 32 GB of internal storage per unit: for event logs and trends up to 6 months
  • Software:
    • Round-robin monitoring mode (Scanning)
    • Local analog audio output

Software Options

  • RF Monitor: RF parameters, SFN (RF Frame Drift), Channel Impulse Response (with TII extraction)
  • Transport & Ensemble Service Monitor: Monitor the description of the multiplex: service/component list

Hardware Options

  • Quad ADV: 4 monitoring Units 4x (RF in) & 2x IP Control in 1 RU
  • Internal GNSSreceiver
  • Dual Power Supply

Default Warranty

  • Standard: 1 YEAR Hardware Warranty and Basic Helpdesk (HW RMA)

Optional Support Services

  • Advance HW + SW 1/3/5 YEAR including:
    • Advance Replacement: hardware warranty with replacement before reception of faulty unit
    • Software Upgrade: access to all software updates & upgrades
    • Priority HelpDesk: priority access to the support helpdesk for any question on product usage, problem request, and change/improvement requests
  • Premium HW 1/3/5 YEAR including:
    • Premium Advance Replacement : hardware warranty with 1 business day replacement before reception of faulty unit (from a dedicated customer spare stock)
  • Premium SW 1/3/5 YEAR including:
    • Premium Software Support: SLA contract manager + guaranteed response time + 2 days per year local proactive visit (training including presentation of new available features, 1 TX site & or HE visit & configuration improvement, discussion on needs for new functionalities)
    • Software Upgrade: access to all software updates & upgrades
    • Priority HelpDesk: priority access to the support helpdesk for any question on product usage, problem request, and change/improvement requests

The ideal tool for accurate & cost-effective monitoring of the quality actually delivered to all points of DAB/DAB+ networks.